[==========] Running 14 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 14 tests from AutoRebalancerTest
[ RUN ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250905 07:47:41.950750 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:41945,127.8.229.125:40721,127.8.229.124:43455
I20250905 07:47:41.952739 9109 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250905 07:47:41.953953 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:41.964663 9117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:41.964727 9114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:41.964680 9115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:41.965356 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:41.966217 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:41.966378 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:41.966461 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058461966439 us; error 0 us; skew 500 ppm
I20250905 07:47:41.966774 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:41.969306 9109 webserver.cc:480] Webserver started at http://127.8.229.126:33441/ using document root <none> and password file <none>
I20250905 07:47:41.969987 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:41.970085 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:41.970484 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:41.973631 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "7f0611409551429ab17e048a8c63df30"
format_stamp: "Formatted at 2025-09-05 07:47:41 on dist-test-slave-pgzt"
I20250905 07:47:41.979422 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20250905 07:47:41.982421 9122 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:41.983841 9109 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250905 07:47:41.984016 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "7f0611409551429ab17e048a8c63df30"
format_stamp: "Formatted at 2025-09-05 07:47:41 on dist-test-slave-pgzt"
I20250905 07:47:41.984194 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:41.997834 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:41.998646 9109 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250905 07:47:41.998924 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:42.018959 9175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:41945 every 8 connection(s)
I20250905 07:47:42.018954 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:41945
I20250905 07:47:42.021420 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:42.024444 9178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:42.024479 9179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:42.025399 9181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.025907 9176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:42.026000 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:42.026300 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:42.026366 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:42.026465 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058462026409 us; error 0 us; skew 500 ppm
I20250905 07:47:42.026643 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:42.027971 9109 webserver.cc:480] Webserver started at http://127.8.229.125:35487/ using document root <none> and password file <none>
I20250905 07:47:42.028210 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:42.028301 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:42.028442 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:42.029104 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root/instance:
uuid: "7970264b39314387a708bfef23125fc9"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.031949 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20250905 07:47:42.033731 9186 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.034114 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:47:42.034233 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root
uuid: "7970264b39314387a708bfef23125fc9"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.034363 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:42.036706 9176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:42.050621 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:42.051510 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:42.054781 9176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } has no permanent_uuid. Determining permanent_uuid...
W20250905 07:47:42.056106 9125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.125:40721: connect: Connection refused (error 111)
W20250905 07:47:42.058284 9176 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.8.229.125:40721: Network error: Client connection negotiation failed: client connection to 127.8.229.125:40721: connect: Connection refused (error 111)
I20250905 07:47:42.068993 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.125:40721
I20250905 07:47:42.069016 9240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.125:40721 every 8 connection(s)
I20250905 07:47:42.070555 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:47:42.070652 9241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250905 07:47:42.073385 9243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:42.073820 9244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.074190 9241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
W20250905 07:47:42.077353 9246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.077723 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:42.078052 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:42.078132 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:42.078213 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058462078176 us; error 0 us; skew 500 ppm
I20250905 07:47:42.078413 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:42.079751 9109 webserver.cc:480] Webserver started at http://127.8.229.124:36209/ using document root <none> and password file <none>
I20250905 07:47:42.080015 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:42.080137 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:42.080278 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:42.081167 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root/instance:
uuid: "b22609af219545fe8353f8a9e588ea6a"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.082536 9241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:42.084414 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20250905 07:47:42.086649 9253 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.087087 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.001s
I20250905 07:47:42.087196 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root
uuid: "b22609af219545fe8353f8a9e588ea6a"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.087369 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:42.090559 9241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } has no permanent_uuid. Determining permanent_uuid...
W20250905 07:47:42.092931 9241 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.8.229.124:43455: Network error: Client connection negotiation failed: client connection to 127.8.229.124:43455: connect: Connection refused (error 111)
I20250905 07:47:42.096791 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:42.097314 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:42.107667 9176 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } attempt: 1
I20250905 07:47:42.112943 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.124:43455
I20250905 07:47:42.113006 9306 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.124:43455 every 8 connection(s)
I20250905 07:47:42.114418 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:47:42.115057 9307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:42.115775 9176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:42.118018 9307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:42.124830 9307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:42.128435 9176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: Bootstrap starting.
I20250905 07:47:42.130537 9307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:42.132953 9176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:42.134523 9176 log.cc:826] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: Log is configured to *not* fsync() on all Append() calls
I20250905 07:47:42.137833 9176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: No bootstrap required, opened a new log
I20250905 07:47:42.138928 9307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a: Bootstrap starting.
I20250905 07:47:42.140389 9307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:42.142113 9307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a: No bootstrap required, opened a new log
I20250905 07:47:42.143404 9307 raft_consensus.cc:357] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.143409 9176 raft_consensus.cc:357] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.143797 9307 raft_consensus.cc:383] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:42.143795 9176 raft_consensus.cc:383] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:42.143934 9307 raft_consensus.cc:738] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b22609af219545fe8353f8a9e588ea6a, State: Initialized, Role: FOLLOWER
I20250905 07:47:42.143956 9176 raft_consensus.cc:738] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0611409551429ab17e048a8c63df30, State: Initialized, Role: FOLLOWER
I20250905 07:47:42.144814 9307 consensus_queue.cc:260] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.144814 9176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.145112 9241 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } attempt: 1
I20250905 07:47:42.145968 9313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.146493 9313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:42.147392 9176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:47:42.148119 9314 sys_catalog.cc:455] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.148415 9307 sys_catalog.cc:564] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:47:42.148458 9314 sys_catalog.cc:458] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:42.162727 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: Bootstrap starting.
W20250905 07:47:42.164578 9330 catalog_manager.cc:1560] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250905 07:47:42.164721 9330 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250905 07:47:42.165776 9241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:42.168278 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250905 07:47:42.168473 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: No bootstrap required, opened a new log
I20250905 07:47:42.169201 9241 raft_consensus.cc:357] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.169452 9241 raft_consensus.cc:383] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:42.169546 9241 raft_consensus.cc:738] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7970264b39314387a708bfef23125fc9, State: Initialized, Role: FOLLOWER
I20250905 07:47:42.169869 9241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
W20250905 07:47:42.169961 9338 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250905 07:47:42.170068 9338 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250905 07:47:42.170724 9241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:47:42.170593 9339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.170854 9339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:42.175707 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250905 07:47:42.176565 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:42.177263 9350 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250905 07:47:42.177382 9350 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250905 07:47:42.180481 9351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:42.180965 9352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:42.181980 9354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.184563 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:42.184813 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:42.184899 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:42.184970 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058462184934 us; error 0 us; skew 500 ppm
I20250905 07:47:42.185141 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:42.186497 9109 webserver.cc:480] Webserver started at http://127.8.229.65:40271/ using document root <none> and password file <none>
I20250905 07:47:42.186728 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:42.186818 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:42.186975 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:42.187706 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "42c27c230caa4aa28a9415168a61845a"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.190660 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20250905 07:47:42.192623 9359 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.193056 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:47:42.193192 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "42c27c230caa4aa28a9415168a61845a"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.193354 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:42.205143 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:42.205950 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:42.206980 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:42.208420 9314 raft_consensus.cc:491] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:42.208729 9314 raft_consensus.cc:513] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.209200 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:42.209359 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.209502 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:42.209599 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.210770 9314 leader_election.cc:290] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f0611409551429ab17e048a8c63df30 (127.8.229.126:41945), 7970264b39314387a708bfef23125fc9 (127.8.229.125:40721)
I20250905 07:47:42.211710 9150 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b22609af219545fe8353f8a9e588ea6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0611409551429ab17e048a8c63df30" is_pre_election: true
I20250905 07:47:42.211685 9216 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b22609af219545fe8353f8a9e588ea6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7970264b39314387a708bfef23125fc9" is_pre_election: true
I20250905 07:47:42.212266 9216 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b22609af219545fe8353f8a9e588ea6a in term 0.
I20250905 07:47:42.212268 9150 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b22609af219545fe8353f8a9e588ea6a in term 0.
I20250905 07:47:42.212910 9257 leader_election.cc:304] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0611409551429ab17e048a8c63df30, b22609af219545fe8353f8a9e588ea6a; no voters:
I20250905 07:47:42.213296 9314 raft_consensus.cc:2802] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:42.213459 9314 raft_consensus.cc:491] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:42.213572 9314 raft_consensus.cc:3058] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:42.215867 9314 raft_consensus.cc:513] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.216408 9314 leader_election.cc:290] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [CANDIDATE]: Term 1 election: Requested vote from peers 7f0611409551429ab17e048a8c63df30 (127.8.229.126:41945), 7970264b39314387a708bfef23125fc9 (127.8.229.125:40721)
I20250905 07:47:42.217058 9150 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b22609af219545fe8353f8a9e588ea6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0611409551429ab17e048a8c63df30"
I20250905 07:47:42.217233 9150 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:42.219522 9150 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b22609af219545fe8353f8a9e588ea6a in term 1.
I20250905 07:47:42.219645 9216 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b22609af219545fe8353f8a9e588ea6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7970264b39314387a708bfef23125fc9"
I20250905 07:47:42.219818 9216 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:42.220134 9257 leader_election.cc:304] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0611409551429ab17e048a8c63df30, b22609af219545fe8353f8a9e588ea6a; no voters:
I20250905 07:47:42.220445 9314 raft_consensus.cc:2802] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:42.223672 9216 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b22609af219545fe8353f8a9e588ea6a in term 1.
I20250905 07:47:42.225152 9314 raft_consensus.cc:695] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 1 LEADER]: Becoming Leader. State: Replica: b22609af219545fe8353f8a9e588ea6a, State: Running, Role: LEADER
I20250905 07:47:42.225814 9314 consensus_queue.cc:237] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } }
I20250905 07:47:42.231333 9370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b22609af219545fe8353f8a9e588ea6a. Latest consensus state: current_term: 1 leader_uuid: "b22609af219545fe8353f8a9e588ea6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.231763 9370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:42.235893 9384 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:47:42.237737 9384 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:47:42.246771 9150 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 1 FOLLOWER]: Refusing update from remote peer b22609af219545fe8353f8a9e588ea6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:42.246771 9216 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 1 FOLLOWER]: Refusing update from remote peer b22609af219545fe8353f8a9e588ea6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:42.247792 9370 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:42.248019 9314 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250905 07:47:42.250869 9128 tablet.cc:2378] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250905 07:47:42.277736 9339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b22609af219545fe8353f8a9e588ea6a. Latest consensus state: current_term: 1 leader_uuid: "b22609af219545fe8353f8a9e588ea6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.278348 9339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:42.279696 9313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b22609af219545fe8353f8a9e588ea6a. Latest consensus state: current_term: 1 leader_uuid: "b22609af219545fe8353f8a9e588ea6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.280033 9313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:42.282410 9390 mvcc.cc:204] Tried to move back new op lower bound from 7196911461352136704 to 7196911461284569088. Current Snapshot: MvccSnapshot[applied={T|T < 7196911461352136704}]
I20250905 07:47:42.294518 9370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b22609af219545fe8353f8a9e588ea6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.294914 9370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:42.295300 9370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b22609af219545fe8353f8a9e588ea6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.295544 9370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:42.297257 9313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b22609af219545fe8353f8a9e588ea6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.297494 9313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:42.302376 9339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b22609af219545fe8353f8a9e588ea6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0611409551429ab17e048a8c63df30" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 41945 } } peers { permanent_uuid: "7970264b39314387a708bfef23125fc9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 40721 } } peers { permanent_uuid: "b22609af219545fe8353f8a9e588ea6a" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 43455 } } }
I20250905 07:47:42.302601 9339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:42.303843 9384 catalog_manager.cc:1349] Generated new cluster ID: 09222a2a0c8841418844eb319e8fe803
I20250905 07:47:42.303962 9384 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:47:42.316627 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:44363
I20250905 07:47:42.316655 9433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:44363 every 8 connection(s)
I20250905 07:47:42.332772 9434 heartbeater.cc:344] Connected to a master server at 127.8.229.124:43455
I20250905 07:47:42.333137 9434 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.334494 9434 heartbeater.cc:507] Master 127.8.229.124:43455 requested a full tablet report, sending...
I20250905 07:47:42.338449 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:47:42.338615 9272 ts_manager.cc:194] Registered new tserver with Master: 42c27c230caa4aa28a9415168a61845a (127.8.229.65:44363)
I20250905 07:47:42.341070 9384 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:47:42.342864 9435 heartbeater.cc:344] Connected to a master server at 127.8.229.126:41945
I20250905 07:47:42.343485 9435 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.344971 9384 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:47:42.345024 9435 heartbeater.cc:507] Master 127.8.229.126:41945 requested a full tablet report, sending...
I20250905 07:47:42.346691 9139 ts_manager.cc:194] Registered new tserver with Master: 42c27c230caa4aa28a9415168a61845a (127.8.229.65:44363)
I20250905 07:47:42.347223 9436 heartbeater.cc:344] Connected to a master server at 127.8.229.125:40721
I20250905 07:47:42.347383 9436 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.347971 9436 heartbeater.cc:507] Master 127.8.229.125:40721 requested a full tablet report, sending...
I20250905 07:47:42.349361 9206 ts_manager.cc:194] Registered new tserver with Master: 42c27c230caa4aa28a9415168a61845a (127.8.229.65:44363)
W20250905 07:47:42.351912 9440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:42.352470 9441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.357089 9109 server_base.cc:1047] running on GCE node
W20250905 07:47:42.357391 9443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.357913 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:42.358057 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:42.358204 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058462358156 us; error 0 us; skew 500 ppm
I20250905 07:47:42.358455 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:42.360056 9109 webserver.cc:480] Webserver started at http://127.8.229.66:38673/ using document root <none> and password file <none>
I20250905 07:47:42.360339 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:42.360847 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:42.361019 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:42.361869 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "9465d297ae2c492aadee597e4ba952f7"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.363868 9384 catalog_manager.cc:5955] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a: Generated new TSK 0
I20250905 07:47:42.365145 9384 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:47:42.365482 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250905 07:47:42.368476 9448 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.368875 9109 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:47:42.368994 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "9465d297ae2c492aadee597e4ba952f7"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.369149 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:42.383134 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:42.383843 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:42.384516 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:42.385468 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:42.385548 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.385669 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:42.385731 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.406307 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:45815
I20250905 07:47:42.406317 9511 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:45815 every 8 connection(s)
I20250905 07:47:42.430781 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:47:42.433111 9512 heartbeater.cc:344] Connected to a master server at 127.8.229.124:43455
I20250905 07:47:42.433291 9512 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.433991 9512 heartbeater.cc:507] Master 127.8.229.124:43455 requested a full tablet report, sending...
W20250905 07:47:42.435180 9518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.435912 9272 ts_manager.cc:194] Registered new tserver with Master: 9465d297ae2c492aadee597e4ba952f7 (127.8.229.66:45815)
I20250905 07:47:42.437511 9513 heartbeater.cc:344] Connected to a master server at 127.8.229.126:41945
I20250905 07:47:42.437664 9513 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.438261 9513 heartbeater.cc:507] Master 127.8.229.126:41945 requested a full tablet report, sending...
I20250905 07:47:42.439165 9272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46306
I20250905 07:47:42.440793 9139 ts_manager.cc:194] Registered new tserver with Master: 9465d297ae2c492aadee597e4ba952f7 (127.8.229.66:45815)
W20250905 07:47:42.444171 9519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:42.444258 9522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:42.444365 9514 heartbeater.cc:344] Connected to a master server at 127.8.229.125:40721
I20250905 07:47:42.444552 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:42.444674 9514 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.444999 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250905 07:47:42.445078 9514 heartbeater.cc:507] Master 127.8.229.125:40721 requested a full tablet report, sending...
W20250905 07:47:42.445106 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:42.445200 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058462445174 us; error 0 us; skew 500 ppm
I20250905 07:47:42.445405 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:42.446283 9206 ts_manager.cc:194] Registered new tserver with Master: 9465d297ae2c492aadee597e4ba952f7 (127.8.229.66:45815)
I20250905 07:47:42.446787 9109 webserver.cc:480] Webserver started at http://127.8.229.67:43959/ using document root <none> and password file <none>
I20250905 07:47:42.447039 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:42.447131 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:42.447302 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:42.448022 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "f49cf7336ec048dd951a87f04a2fdbd2"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.450822 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:47:42.452600 9526 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.452996 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:47:42.453102 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "f49cf7336ec048dd951a87f04a2fdbd2"
format_stamp: "Formatted at 2025-09-05 07:47:42 on dist-test-slave-pgzt"
I20250905 07:47:42.453239 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:42.476621 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:42.477303 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:42.478040 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:42.479146 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:42.479239 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.479359 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:42.479490 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:42.503510 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:45547
I20250905 07:47:42.503559 9589 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:45547 every 8 connection(s)
I20250905 07:47:42.525210 9590 heartbeater.cc:344] Connected to a master server at 127.8.229.124:43455
I20250905 07:47:42.525383 9590 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.525957 9590 heartbeater.cc:507] Master 127.8.229.124:43455 requested a full tablet report, sending...
I20250905 07:47:42.527401 9272 ts_manager.cc:194] Registered new tserver with Master: f49cf7336ec048dd951a87f04a2fdbd2 (127.8.229.67:45547)
I20250905 07:47:42.528560 9272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46310
I20250905 07:47:42.531941 9591 heartbeater.cc:344] Connected to a master server at 127.8.229.126:41945
I20250905 07:47:42.532101 9591 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.532461 9591 heartbeater.cc:507] Master 127.8.229.126:41945 requested a full tablet report, sending...
I20250905 07:47:42.533593 9139 ts_manager.cc:194] Registered new tserver with Master: f49cf7336ec048dd951a87f04a2fdbd2 (127.8.229.67:45547)
I20250905 07:47:42.534315 9592 heartbeater.cc:344] Connected to a master server at 127.8.229.125:40721
I20250905 07:47:42.534459 9592 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:42.534732 9592 heartbeater.cc:507] Master 127.8.229.125:40721 requested a full tablet report, sending...
I20250905 07:47:42.535748 9206 ts_manager.cc:194] Registered new tserver with Master: f49cf7336ec048dd951a87f04a2fdbd2 (127.8.229.67:45547)
I20250905 07:47:42.536248 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017549361s
I20250905 07:47:43.171917 9338 catalog_manager.cc:1261] Loaded cluster ID: 09222a2a0c8841418844eb319e8fe803
I20250905 07:47:43.172035 9338 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: loading cluster ID for follower catalog manager: success
I20250905 07:47:43.174206 9338 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: acquiring CA information for follower catalog manager: success
I20250905 07:47:43.175420 9338 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250905 07:47:43.178361 9350 catalog_manager.cc:1261] Loaded cluster ID: 09222a2a0c8841418844eb319e8fe803
I20250905 07:47:43.178452 9350 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: loading cluster ID for follower catalog manager: success
I20250905 07:47:43.180279 9350 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: acquiring CA information for follower catalog manager: success
I20250905 07:47:43.181247 9350 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250905 07:47:43.341850 9272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46294
I20250905 07:47:43.442749 9512 heartbeater.cc:499] Master 127.8.229.124:43455 was elected leader, sending a full tablet report...
I20250905 07:47:43.531018 9590 heartbeater.cc:499] Master 127.8.229.124:43455 was elected leader, sending a full tablet report...
I20250905 07:47:44.158460 9329 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:47:44.343194 9434 heartbeater.cc:499] Master 127.8.229.124:43455 was elected leader, sending a full tablet report...
I20250905 07:47:44.596637 9109 test_util.cc:276] Using random seed: 228704883
I20250905 07:47:44.635147 9272 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46312:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250905 07:47:44.678006 9389 tablet_service.cc:1468] Processing CreateTablet for tablet ed41101dad9746ea869b363defdf1e07 (DEFAULT_TABLE table=test-workload [id=828b4b3367704a5d8e3232f9de82df83]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:47:44.678041 9391 tablet_service.cc:1468] Processing CreateTablet for tablet aa78b0f518e44436a65b47bb6663bade (DEFAULT_TABLE table=test-workload [id=828b4b3367704a5d8e3232f9de82df83]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:47:44.679109 9391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa78b0f518e44436a65b47bb6663bade. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:44.679975 9389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed41101dad9746ea869b363defdf1e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:44.681165 9477 tablet_service.cc:1468] Processing CreateTablet for tablet 2b707458176e4f2d9392c6c07851c53d (DEFAULT_TABLE table=test-workload [id=828b4b3367704a5d8e3232f9de82df83]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:47:44.681803 9477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b707458176e4f2d9392c6c07851c53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:44.684810 9555 tablet_service.cc:1468] Processing CreateTablet for tablet ac977868e6f04058b8449bebd679c611 (DEFAULT_TABLE table=test-workload [id=828b4b3367704a5d8e3232f9de82df83]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:47:44.685401 9555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac977868e6f04058b8449bebd679c611. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:44.689702 9625 tablet_bootstrap.cc:492] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7: Bootstrap starting.
I20250905 07:47:44.692011 9625 tablet_bootstrap.cc:654] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:44.693559 9624 tablet_bootstrap.cc:492] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a: Bootstrap starting.
I20250905 07:47:44.697023 9624 tablet_bootstrap.cc:654] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:44.697049 9627 tablet_bootstrap.cc:492] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2: Bootstrap starting.
I20250905 07:47:44.698630 9625 tablet_bootstrap.cc:492] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7: No bootstrap required, opened a new log
I20250905 07:47:44.698897 9625 ts_tablet_manager.cc:1397] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.005s
I20250905 07:47:44.699604 9627 tablet_bootstrap.cc:654] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:44.699797 9624 tablet_bootstrap.cc:492] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a: No bootstrap required, opened a new log
I20250905 07:47:44.699852 9625 raft_consensus.cc:357] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9465d297ae2c492aadee597e4ba952f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 45815 } }
I20250905 07:47:44.699968 9624 ts_tablet_manager.cc:1397] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20250905 07:47:44.700071 9625 raft_consensus.cc:383] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:44.700196 9625 raft_consensus.cc:738] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9465d297ae2c492aadee597e4ba952f7, State: Initialized, Role: FOLLOWER
I20250905 07:47:44.700525 9625 consensus_queue.cc:260] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9465d297ae2c492aadee597e4ba952f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 45815 } }
I20250905 07:47:44.700788 9625 raft_consensus.cc:397] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:44.700820 9624 raft_consensus.cc:357] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.700945 9625 raft_consensus.cc:491] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:44.701038 9624 raft_consensus.cc:383] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:44.701102 9625 raft_consensus.cc:3058] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:44.701241 9624 raft_consensus.cc:738] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c27c230caa4aa28a9415168a61845a, State: Initialized, Role: FOLLOWER
I20250905 07:47:44.701519 9624 consensus_queue.cc:260] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.701707 9624 raft_consensus.cc:397] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:44.701800 9624 raft_consensus.cc:491] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:44.701961 9624 raft_consensus.cc:3058] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:44.701987 9627 tablet_bootstrap.cc:492] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2: No bootstrap required, opened a new log
I20250905 07:47:44.702136 9627 ts_tablet_manager.cc:1397] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20250905 07:47:44.702759 9627 raft_consensus.cc:357] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49cf7336ec048dd951a87f04a2fdbd2" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45547 } }
I20250905 07:47:44.702912 9627 raft_consensus.cc:383] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:44.703042 9627 raft_consensus.cc:738] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49cf7336ec048dd951a87f04a2fdbd2, State: Initialized, Role: FOLLOWER
I20250905 07:47:44.703280 9627 consensus_queue.cc:260] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49cf7336ec048dd951a87f04a2fdbd2" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45547 } }
I20250905 07:47:44.703485 9627 raft_consensus.cc:397] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:44.703581 9627 raft_consensus.cc:491] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:44.703701 9627 raft_consensus.cc:3058] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:44.704404 9625 raft_consensus.cc:513] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9465d297ae2c492aadee597e4ba952f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 45815 } }
I20250905 07:47:44.704680 9625 leader_election.cc:304] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9465d297ae2c492aadee597e4ba952f7; no voters:
I20250905 07:47:44.704898 9624 raft_consensus.cc:513] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.705255 9624 leader_election.cc:304] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c27c230caa4aa28a9415168a61845a; no voters:
I20250905 07:47:44.705862 9624 leader_election.cc:290] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:44.706010 9631 raft_consensus.cc:2802] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:44.706844 9627 raft_consensus.cc:513] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49cf7336ec048dd951a87f04a2fdbd2" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45547 } }
I20250905 07:47:44.707252 9627 leader_election.cc:304] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f49cf7336ec048dd951a87f04a2fdbd2; no voters:
I20250905 07:47:44.708681 9634 raft_consensus.cc:2802] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:44.708894 9627 leader_election.cc:290] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:44.715402 9634 raft_consensus.cc:695] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 1 LEADER]: Becoming Leader. State: Replica: f49cf7336ec048dd951a87f04a2fdbd2, State: Running, Role: LEADER
I20250905 07:47:44.715967 9625 leader_election.cc:290] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:44.716944 9634 consensus_queue.cc:237] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49cf7336ec048dd951a87f04a2fdbd2" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45547 } }
I20250905 07:47:44.719211 9627 ts_tablet_manager.cc:1428] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2: Time spent starting tablet: real 0.017s user 0.003s sys 0.006s
I20250905 07:47:44.716130 9630 raft_consensus.cc:2802] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:44.722779 9631 raft_consensus.cc:695] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 1 LEADER]: Becoming Leader. State: Replica: 42c27c230caa4aa28a9415168a61845a, State: Running, Role: LEADER
I20250905 07:47:44.723212 9624 ts_tablet_manager.cc:1428] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a: Time spent starting tablet: real 0.023s user 0.008s sys 0.015s
I20250905 07:47:44.723286 9631 consensus_queue.cc:237] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.723970 9625 ts_tablet_manager.cc:1428] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7: Time spent starting tablet: real 0.025s user 0.014s sys 0.010s
I20250905 07:47:44.727241 9272 catalog_manager.cc:5582] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a reported cstate change: term changed from 0 to 1, leader changed from <none> to 42c27c230caa4aa28a9415168a61845a (127.8.229.65). New cstate: current_term: 1 leader_uuid: "42c27c230caa4aa28a9415168a61845a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:44.727816 9630 raft_consensus.cc:695] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 1 LEADER]: Becoming Leader. State: Replica: 9465d297ae2c492aadee597e4ba952f7, State: Running, Role: LEADER
I20250905 07:47:44.728137 9630 consensus_queue.cc:237] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9465d297ae2c492aadee597e4ba952f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 45815 } }
I20250905 07:47:44.729077 9624 tablet_bootstrap.cc:492] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a: Bootstrap starting.
I20250905 07:47:44.735175 9624 tablet_bootstrap.cc:654] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:44.746668 9271 catalog_manager.cc:5582] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9465d297ae2c492aadee597e4ba952f7 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "9465d297ae2c492aadee597e4ba952f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9465d297ae2c492aadee597e4ba952f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 45815 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:44.748109 9270 catalog_manager.cc:5582] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f49cf7336ec048dd951a87f04a2fdbd2 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "f49cf7336ec048dd951a87f04a2fdbd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49cf7336ec048dd951a87f04a2fdbd2" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45547 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:44.758724 9624 tablet_bootstrap.cc:492] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a: No bootstrap required, opened a new log
I20250905 07:47:44.758920 9624 ts_tablet_manager.cc:1397] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.000s
I20250905 07:47:44.759603 9624 raft_consensus.cc:357] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.759749 9624 raft_consensus.cc:383] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:44.759830 9624 raft_consensus.cc:738] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c27c230caa4aa28a9415168a61845a, State: Initialized, Role: FOLLOWER
I20250905 07:47:44.760053 9624 consensus_queue.cc:260] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.760183 9624 raft_consensus.cc:397] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:44.760282 9624 raft_consensus.cc:491] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:44.760377 9624 raft_consensus.cc:3058] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:44.762207 9624 raft_consensus.cc:513] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.762392 9624 leader_election.cc:304] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c27c230caa4aa28a9415168a61845a; no voters:
I20250905 07:47:44.762583 9624 leader_election.cc:290] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:44.762997 9624 ts_tablet_manager.cc:1428] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250905 07:47:44.762727 9642 raft_consensus.cc:2802] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:44.763234 9642 raft_consensus.cc:695] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 1 LEADER]: Becoming Leader. State: Replica: 42c27c230caa4aa28a9415168a61845a, State: Running, Role: LEADER
I20250905 07:47:44.763509 9642 consensus_queue.cc:237] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } }
I20250905 07:47:44.765929 9272 catalog_manager.cc:5582] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a reported cstate change: term changed from 0 to 1, leader changed from <none> to 42c27c230caa4aa28a9415168a61845a (127.8.229.65). New cstate: current_term: 1 leader_uuid: "42c27c230caa4aa28a9415168a61845a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c27c230caa4aa28a9415168a61845a" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44363 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:44.778836 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:47:44.789083 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:44.789506 9109 tablet_replica.cc:331] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a: stopping tablet replica
I20250905 07:47:44.789932 9109 raft_consensus.cc:2241] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:44.790222 9109 raft_consensus.cc:2270] T ed41101dad9746ea869b363defdf1e07 P 42c27c230caa4aa28a9415168a61845a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:44.791822 9109 tablet_replica.cc:331] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a: stopping tablet replica
I20250905 07:47:44.791980 9109 raft_consensus.cc:2241] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:44.792142 9109 raft_consensus.cc:2270] T aa78b0f518e44436a65b47bb6663bade P 42c27c230caa4aa28a9415168a61845a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:44.807994 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:47:44.813449 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:47:44.821682 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:44.822103 9109 tablet_replica.cc:331] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7: stopping tablet replica
I20250905 07:47:44.822329 9109 raft_consensus.cc:2241] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:44.822535 9109 raft_consensus.cc:2270] T 2b707458176e4f2d9392c6c07851c53d P 9465d297ae2c492aadee597e4ba952f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:44.826995 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:47:44.829672 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:47:44.838546 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:44.838948 9109 tablet_replica.cc:331] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2: stopping tablet replica
I20250905 07:47:44.839135 9109 raft_consensus.cc:2241] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:44.839324 9109 raft_consensus.cc:2270] T ac977868e6f04058b8449bebd679c611 P f49cf7336ec048dd951a87f04a2fdbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:44.843544 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:47:44.846240 9109 master.cc:561] Master@127.8.229.126:41945 shutting down...
I20250905 07:47:44.855132 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:44.855386 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:44.855607 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 7f0611409551429ab17e048a8c63df30: stopping tablet replica
I20250905 07:47:44.870293 9109 master.cc:583] Master@127.8.229.126:41945 shutdown complete.
I20250905 07:47:44.875094 9109 master.cc:561] Master@127.8.229.125:40721 shutting down...
I20250905 07:47:44.882489 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:44.882720 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:44.882844 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 7970264b39314387a708bfef23125fc9: stopping tablet replica
I20250905 07:47:44.896813 9109 master.cc:583] Master@127.8.229.125:40721 shutdown complete.
I20250905 07:47:44.900915 9109 master.cc:561] Master@127.8.229.124:43455 shutting down...
I20250905 07:47:44.907572 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:44.908030 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:44.908151 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P b22609af219545fe8353f8a9e588ea6a: stopping tablet replica
I20250905 07:47:44.922663 9109 master.cc:583] Master@127.8.229.124:43455 shutdown complete.
[ OK ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing (2988 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn
I20250905 07:47:44.936794 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:36343
I20250905 07:47:44.937371 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:44.939802 9649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:44.939868 9650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:44.940572 9652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:44.941304 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:44.941546 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:44.941622 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:44.941679 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058464941660 us; error 0 us; skew 500 ppm
I20250905 07:47:44.941851 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:44.942961 9109 webserver.cc:480] Webserver started at http://127.8.229.126:45389/ using document root <none> and password file <none>
I20250905 07:47:44.943205 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:44.943282 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:44.943472 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:44.944172 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "ba6f3153563043fe860edb6e045f4055"
format_stamp: "Formatted at 2025-09-05 07:47:44 on dist-test-slave-pgzt"
I20250905 07:47:44.948845 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20250905 07:47:44.950445 9657 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:44.950773 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:47:44.950879 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "ba6f3153563043fe860edb6e045f4055"
format_stamp: "Formatted at 2025-09-05 07:47:44 on dist-test-slave-pgzt"
I20250905 07:47:44.951011 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:44.971586 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:44.972188 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:44.986584 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:36343
I20250905 07:47:44.986608 9708 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:36343 every 8 connection(s)
I20250905 07:47:44.987963 9709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:44.991842 9709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055: Bootstrap starting.
I20250905 07:47:44.993202 9709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:44.994783 9709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055: No bootstrap required, opened a new log
I20250905 07:47:44.995489 9709 raft_consensus.cc:357] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6f3153563043fe860edb6e045f4055" member_type: VOTER }
I20250905 07:47:44.995632 9709 raft_consensus.cc:383] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:44.995741 9709 raft_consensus.cc:738] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6f3153563043fe860edb6e045f4055, State: Initialized, Role: FOLLOWER
I20250905 07:47:44.995947 9709 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6f3153563043fe860edb6e045f4055" member_type: VOTER }
I20250905 07:47:44.996096 9709 raft_consensus.cc:397] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:44.996191 9709 raft_consensus.cc:491] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:44.996307 9709 raft_consensus.cc:3058] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:44.998214 9709 raft_consensus.cc:513] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6f3153563043fe860edb6e045f4055" member_type: VOTER }
I20250905 07:47:44.998492 9709 leader_election.cc:304] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba6f3153563043fe860edb6e045f4055; no voters:
I20250905 07:47:44.999503 9709 leader_election.cc:290] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:44.999636 9712 raft_consensus.cc:2802] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:45.000171 9712 raft_consensus.cc:695] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 1 LEADER]: Becoming Leader. State: Replica: ba6f3153563043fe860edb6e045f4055, State: Running, Role: LEADER
I20250905 07:47:45.000452 9712 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6f3153563043fe860edb6e045f4055" member_type: VOTER }
I20250905 07:47:45.000502 9709 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:47:45.001358 9713 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba6f3153563043fe860edb6e045f4055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6f3153563043fe860edb6e045f4055" member_type: VOTER } }
I20250905 07:47:45.001605 9713 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:45.001390 9714 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba6f3153563043fe860edb6e045f4055. Latest consensus state: current_term: 1 leader_uuid: "ba6f3153563043fe860edb6e045f4055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6f3153563043fe860edb6e045f4055" member_type: VOTER } }
I20250905 07:47:45.002013 9714 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:45.008680 9717 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:47:45.010488 9717 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:47:45.011368 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:47:45.013563 9717 catalog_manager.cc:1349] Generated new cluster ID: be8d2639dbdf49c39824a72ecb18212b
I20250905 07:47:45.013634 9717 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:47:45.040627 9717 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:47:45.041416 9717 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:47:45.052291 9717 catalog_manager.cc:5955] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055: Generated new TSK 0
I20250905 07:47:45.052523 9717 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:47:45.076885 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:45.080309 9731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:45.080408 9732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:45.081020 9734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:45.081480 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:45.081733 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:45.081804 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:45.081897 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058465081856 us; error 0 us; skew 500 ppm
I20250905 07:47:45.082067 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:45.083246 9109 webserver.cc:480] Webserver started at http://127.8.229.65:44605/ using document root <none> and password file <none>
I20250905 07:47:45.083523 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:45.083627 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:45.083772 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:45.084462 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "6cd0127a22db4e5b83195785b0f73972"
format_stamp: "Formatted at 2025-09-05 07:47:45 on dist-test-slave-pgzt"
I20250905 07:47:45.086859 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20250905 07:47:45.088493 9739 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.088835 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:47:45.088943 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "6cd0127a22db4e5b83195785b0f73972"
format_stamp: "Formatted at 2025-09-05 07:47:45 on dist-test-slave-pgzt"
I20250905 07:47:45.089103 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:45.138549 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:45.139195 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:45.139858 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:45.140760 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:45.140836 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.140977 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:45.141038 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.157979 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:38187
I20250905 07:47:45.157999 9801 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:38187 every 8 connection(s)
I20250905 07:47:45.159731 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:45.162732 9806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:45.162931 9807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:45.164947 9809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:45.165258 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:45.165602 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:45.165730 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:45.165810 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058465165786 us; error 0 us; skew 500 ppm
I20250905 07:47:45.165999 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:45.167389 9109 webserver.cc:480] Webserver started at http://127.8.229.66:46565/ using document root <none> and password file <none>
I20250905 07:47:45.167658 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:45.167732 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:45.167846 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:45.168543 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "e835801287e04854a5a9bbcbd764a087"
format_stamp: "Formatted at 2025-09-05 07:47:45 on dist-test-slave-pgzt"
I20250905 07:47:45.171087 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20250905 07:47:45.172641 9814 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.172986 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:47:45.173087 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "e835801287e04854a5a9bbcbd764a087"
format_stamp: "Formatted at 2025-09-05 07:47:45 on dist-test-slave-pgzt"
I20250905 07:47:45.173200 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:45.175611 9802 heartbeater.cc:344] Connected to a master server at 127.8.229.126:36343
I20250905 07:47:45.175752 9802 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:45.176147 9802 heartbeater.cc:507] Master 127.8.229.126:36343 requested a full tablet report, sending...
I20250905 07:47:45.177086 9674 ts_manager.cc:194] Registered new tserver with Master: 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187)
I20250905 07:47:45.178623 9674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57300
I20250905 07:47:45.196053 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:45.196892 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:45.197728 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:45.198901 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:45.198994 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.199070 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:45.199126 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.213315 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:44905
I20250905 07:47:45.213339 9876 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:44905 every 8 connection(s)
I20250905 07:47:45.215056 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:45.218178 9880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:45.218544 9881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:45.220897 9883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:45.221801 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:45.222266 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:45.222360 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:45.222397 9877 heartbeater.cc:344] Connected to a master server at 127.8.229.126:36343
I20250905 07:47:45.222457 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058465222412 us; error 0 us; skew 500 ppm
I20250905 07:47:45.222548 9877 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:45.222635 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:45.222980 9877 heartbeater.cc:507] Master 127.8.229.126:36343 requested a full tablet report, sending...
I20250905 07:47:45.223932 9109 webserver.cc:480] Webserver started at http://127.8.229.67:39127/ using document root <none> and password file <none>
I20250905 07:47:45.224043 9674 ts_manager.cc:194] Registered new tserver with Master: e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905)
I20250905 07:47:45.224184 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:45.224291 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:45.224424 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:45.224895 9674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57316
I20250905 07:47:45.225075 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "afcca3d30c72404cbcfb08328bdb8efe"
format_stamp: "Formatted at 2025-09-05 07:47:45 on dist-test-slave-pgzt"
I20250905 07:47:45.227702 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:47:45.229242 9888 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.229645 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:47:45.229736 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "afcca3d30c72404cbcfb08328bdb8efe"
format_stamp: "Formatted at 2025-09-05 07:47:45 on dist-test-slave-pgzt"
I20250905 07:47:45.229848 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:45.246362 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:45.246953 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:45.247553 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:45.248395 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:45.248461 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.248548 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:45.248592 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:45.262010 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:36149
I20250905 07:47:45.262043 9950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:36149 every 8 connection(s)
I20250905 07:47:45.269212 9951 heartbeater.cc:344] Connected to a master server at 127.8.229.126:36343
I20250905 07:47:45.269382 9951 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:45.269783 9951 heartbeater.cc:507] Master 127.8.229.126:36343 requested a full tablet report, sending...
I20250905 07:47:45.270605 9674 ts_manager.cc:194] Registered new tserver with Master: afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:45.270844 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007875888s
I20250905 07:47:45.271348 9674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57328
I20250905 07:47:46.179955 9802 heartbeater.cc:499] Master 127.8.229.126:36343 was elected leader, sending a full tablet report...
I20250905 07:47:46.226291 9877 heartbeater.cc:499] Master 127.8.229.126:36343 was elected leader, sending a full tablet report...
I20250905 07:47:46.272758 9951 heartbeater.cc:499] Master 127.8.229.126:36343 was elected leader, sending a full tablet report...
I20250905 07:47:46.298693 9109 test_util.cc:276] Using random seed: 230406940
I20250905 07:47:46.310881 9674 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57344:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o""\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:47:46.311633 9674 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:47:46.338692 9767 tablet_service.cc:1468] Processing CreateTablet for tablet 90dabf9a64384373a330c56033e03b9c (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION VALUES < 268435455
I20250905 07:47:46.339102 9764 tablet_service.cc:1468] Processing CreateTablet for tablet eda10554d4b44034b1b97c55315e5e0f (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20250905 07:47:46.339361 9765 tablet_service.cc:1468] Processing CreateTablet for tablet eb3bc705b77c4e30a481e3b291ea2ee4 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20250905 07:47:46.338697 9766 tablet_service.cc:1468] Processing CreateTablet for tablet c0ff44f9708145e995065016bf8d8fcd (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20250905 07:47:46.340266 9763 tablet_service.cc:1468] Processing CreateTablet for tablet 1397ac721f3e41ae99dcebddc2bba870 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20250905 07:47:46.341012 9760 tablet_service.cc:1468] Processing CreateTablet for tablet d7afbcc8155649f0b144345c6c807762 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20250905 07:47:46.341275 9761 tablet_service.cc:1468] Processing CreateTablet for tablet d13057813c0944f494e40ab24a188261 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20250905 07:47:46.339365 9767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90dabf9a64384373a330c56033e03b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.341665 9764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eda10554d4b44034b1b97c55315e5e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.341986 9763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1397ac721f3e41ae99dcebddc2bba870. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.342051 9762 tablet_service.cc:1468] Processing CreateTablet for tablet 3d4952e38a8e4aefa32a14640633e595 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20250905 07:47:46.342201 9760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7afbcc8155649f0b144345c6c807762. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.342532 9762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4952e38a8e4aefa32a14640633e595. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.342998 9766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ff44f9708145e995065016bf8d8fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.347580 9765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb3bc705b77c4e30a481e3b291ea2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.347847 9761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d13057813c0944f494e40ab24a188261. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.357462 9841 tablet_service.cc:1468] Processing CreateTablet for tablet c0ff44f9708145e995065016bf8d8fcd (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20250905 07:47:46.358688 9842 tablet_service.cc:1468] Processing CreateTablet for tablet 90dabf9a64384373a330c56033e03b9c (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION VALUES < 268435455
I20250905 07:47:46.359189 9842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90dabf9a64384373a330c56033e03b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.360512 9840 tablet_service.cc:1468] Processing CreateTablet for tablet eb3bc705b77c4e30a481e3b291ea2ee4 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20250905 07:47:46.361064 9838 tablet_service.cc:1468] Processing CreateTablet for tablet 1397ac721f3e41ae99dcebddc2bba870 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20250905 07:47:46.362062 9841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ff44f9708145e995065016bf8d8fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.365010 9837 tablet_service.cc:1468] Processing CreateTablet for tablet 3d4952e38a8e4aefa32a14640633e595 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20250905 07:47:46.365926 9839 tablet_service.cc:1468] Processing CreateTablet for tablet eda10554d4b44034b1b97c55315e5e0f (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20250905 07:47:46.365985 9835 tablet_service.cc:1468] Processing CreateTablet for tablet d7afbcc8155649f0b144345c6c807762 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20250905 07:47:46.366870 9838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1397ac721f3e41ae99dcebddc2bba870. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.367290 9836 tablet_service.cc:1468] Processing CreateTablet for tablet d13057813c0944f494e40ab24a188261 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20250905 07:47:46.369370 9839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eda10554d4b44034b1b97c55315e5e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.370229 9840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb3bc705b77c4e30a481e3b291ea2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.371037 9836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d13057813c0944f494e40ab24a188261. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.371541 9837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4952e38a8e4aefa32a14640633e595. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.371779 9835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7afbcc8155649f0b144345c6c807762. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.373354 9971 tablet_bootstrap.cc:492] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.376410 9910 tablet_service.cc:1468] Processing CreateTablet for tablet d13057813c0944f494e40ab24a188261 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20250905 07:47:46.377064 9916 tablet_service.cc:1468] Processing CreateTablet for tablet 90dabf9a64384373a330c56033e03b9c (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION VALUES < 268435455
I20250905 07:47:46.377708 9910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d13057813c0944f494e40ab24a188261. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.378727 9913 tablet_service.cc:1468] Processing CreateTablet for tablet eda10554d4b44034b1b97c55315e5e0f (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20250905 07:47:46.378926 9916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90dabf9a64384373a330c56033e03b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.379734 9909 tablet_service.cc:1468] Processing CreateTablet for tablet d7afbcc8155649f0b144345c6c807762 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20250905 07:47:46.380823 9914 tablet_service.cc:1468] Processing CreateTablet for tablet eb3bc705b77c4e30a481e3b291ea2ee4 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20250905 07:47:46.381080 9912 tablet_service.cc:1468] Processing CreateTablet for tablet 1397ac721f3e41ae99dcebddc2bba870 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20250905 07:47:46.381060 9911 tablet_service.cc:1468] Processing CreateTablet for tablet 3d4952e38a8e4aefa32a14640633e595 (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20250905 07:47:46.383476 9915 tablet_service.cc:1468] Processing CreateTablet for tablet c0ff44f9708145e995065016bf8d8fcd (DEFAULT_TABLE table=test-workload [id=c739ac1583e84dafa1ce75c906c44b94]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20250905 07:47:46.387871 9913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eda10554d4b44034b1b97c55315e5e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.388710 9909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7afbcc8155649f0b144345c6c807762. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.389596 9912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1397ac721f3e41ae99dcebddc2bba870. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.390388 9911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4952e38a8e4aefa32a14640633e595. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.391369 9914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb3bc705b77c4e30a481e3b291ea2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.393086 9915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ff44f9708145e995065016bf8d8fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:46.396142 9972 tablet_bootstrap.cc:492] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.401309 9971 tablet_bootstrap.cc:654] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.418251 9972 tablet_bootstrap.cc:654] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.428009 9974 tablet_bootstrap.cc:492] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.435477 9971 tablet_bootstrap.cc:492] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.435672 9971 ts_tablet_manager.cc:1397] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.063s user 0.005s sys 0.011s
I20250905 07:47:46.435896 9974 tablet_bootstrap.cc:654] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.436491 9971 raft_consensus.cc:357] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.436688 9971 raft_consensus.cc:383] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.436779 9971 raft_consensus.cc:738] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.437045 9971 consensus_queue.cc:260] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.437906 9974 tablet_bootstrap.cc:492] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.438066 9974 ts_tablet_manager.cc:1397] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20250905 07:47:46.438072 9971 ts_tablet_manager.cc:1428] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:47:46.438263 9972 tablet_bootstrap.cc:492] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.438417 9972 ts_tablet_manager.cc:1397] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.005s
I20250905 07:47:46.438639 9971 tablet_bootstrap.cc:492] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.438879 9974 raft_consensus.cc:357] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.439083 9974 raft_consensus.cc:383] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.439218 9974 raft_consensus.cc:738] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.439145 9972 raft_consensus.cc:357] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.439333 9972 raft_consensus.cc:383] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.439613 9974 consensus_queue.cc:260] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.439922 9972 raft_consensus.cc:738] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.440248 9972 consensus_queue.cc:260] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.441146 9971 tablet_bootstrap.cc:654] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.443420 9974 ts_tablet_manager.cc:1428] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20250905 07:47:46.443706 9974 tablet_bootstrap.cc:492] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.445508 9974 tablet_bootstrap.cc:654] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.447393 9972 ts_tablet_manager.cc:1428] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.009s user 0.000s sys 0.008s
I20250905 07:47:46.447824 9972 tablet_bootstrap.cc:492] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.449632 9971 tablet_bootstrap.cc:492] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.449759 9972 tablet_bootstrap.cc:654] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.449911 9971 ts_tablet_manager.cc:1397] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20250905 07:47:46.450917 9971 raft_consensus.cc:357] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.451254 9971 raft_consensus.cc:383] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.451409 9971 raft_consensus.cc:738] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.451857 9971 consensus_queue.cc:260] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.452498 9971 ts_tablet_manager.cc:1428] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:47:46.452876 9971 tablet_bootstrap.cc:492] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.454089 9974 tablet_bootstrap.cc:492] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.454633 9974 ts_tablet_manager.cc:1397] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20250905 07:47:46.455468 9971 tablet_bootstrap.cc:654] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.455616 9974 raft_consensus.cc:357] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.455963 9974 raft_consensus.cc:383] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.456099 9974 raft_consensus.cc:738] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.456504 9974 consensus_queue.cc:260] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.456902 9974 ts_tablet_manager.cc:1428] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:47:46.457271 9974 tablet_bootstrap.cc:492] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.459064 9974 tablet_bootstrap.cc:654] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.461413 9972 tablet_bootstrap.cc:492] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.461621 9972 ts_tablet_manager.cc:1397] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.002s
I20250905 07:47:46.462582 9972 raft_consensus.cc:357] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.462854 9972 raft_consensus.cc:383] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.462945 9972 raft_consensus.cc:738] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.463269 9972 consensus_queue.cc:260] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.463838 9972 ts_tablet_manager.cc:1428] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:47:46.463934 9971 tablet_bootstrap.cc:492] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.464067 9971 ts_tablet_manager.cc:1397] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20250905 07:47:46.464197 9972 tablet_bootstrap.cc:492] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.464816 9971 raft_consensus.cc:357] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.465056 9971 raft_consensus.cc:383] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.465179 9971 raft_consensus.cc:738] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.465493 9971 consensus_queue.cc:260] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.465975 9972 tablet_bootstrap.cc:654] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.466140 9971 ts_tablet_manager.cc:1428] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:47:46.466477 9971 tablet_bootstrap.cc:492] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.468268 9972 tablet_bootstrap.cc:492] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.468300 9971 tablet_bootstrap.cc:654] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.468420 9972 ts_tablet_manager.cc:1397] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20250905 07:47:46.469022 9974 tablet_bootstrap.cc:492] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.469164 9974 ts_tablet_manager.cc:1397] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20250905 07:47:46.469353 9972 raft_consensus.cc:357] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.469527 9972 raft_consensus.cc:383] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.469626 9972 raft_consensus.cc:738] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.469861 9972 consensus_queue.cc:260] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.469951 9974 raft_consensus.cc:357] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.470105 9974 raft_consensus.cc:383] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.470204 9974 raft_consensus.cc:738] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.470427 9972 ts_tablet_manager.cc:1428] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:47:46.470449 9974 consensus_queue.cc:260] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.470700 9972 tablet_bootstrap.cc:492] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.470928 9974 ts_tablet_manager.cc:1428] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:47:46.471171 9974 tablet_bootstrap.cc:492] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.472405 9972 tablet_bootstrap.cc:654] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.475742 9974 tablet_bootstrap.cc:654] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.478085 9974 tablet_bootstrap.cc:492] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.478118 9971 tablet_bootstrap.cc:492] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.478276 9974 ts_tablet_manager.cc:1397] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250905 07:47:46.478328 9971 ts_tablet_manager.cc:1397] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.011s
I20250905 07:47:46.478698 9972 tablet_bootstrap.cc:492] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.478858 9972 ts_tablet_manager.cc:1397] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20250905 07:47:46.479135 9974 raft_consensus.cc:357] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.479305 9974 raft_consensus.cc:383] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.479224 9971 raft_consensus.cc:357] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.479387 9974 raft_consensus.cc:738] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.479609 9971 raft_consensus.cc:383] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.479628 9972 raft_consensus.cc:357] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.479751 9971 raft_consensus.cc:738] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.479727 9974 consensus_queue.cc:260] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.479836 9972 raft_consensus.cc:383] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.479975 9972 raft_consensus.cc:738] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.480317 9974 ts_tablet_manager.cc:1428] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:47:46.480252 9972 consensus_queue.cc:260] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.480655 9974 tablet_bootstrap.cc:492] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.480705 9971 consensus_queue.cc:260] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.481923 9972 ts_tablet_manager.cc:1428] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:47:46.482223 9972 tablet_bootstrap.cc:492] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.482390 9974 tablet_bootstrap.cc:654] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.483414 9971 ts_tablet_manager.cc:1428] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250905 07:47:46.484241 9972 tablet_bootstrap.cc:654] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.484411 9971 tablet_bootstrap.cc:492] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.485643 9979 raft_consensus.cc:491] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.485832 9979 raft_consensus.cc:513] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.486145 9971 tablet_bootstrap.cc:654] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.487175 9979 leader_election.cc:290] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.518894 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d13057813c0944f494e40ab24a188261" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972" is_pre_election: true
I20250905 07:47:46.519407 9777 raft_consensus.cc:2466] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.521114 9817 leader_election.cc:304] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.521520 9979 raft_consensus.cc:2802] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.522042 9979 raft_consensus.cc:491] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.522199 9979 raft_consensus.cc:3058] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.522799 9971 tablet_bootstrap.cc:492] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.522977 9971 ts_tablet_manager.cc:1397] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.039s user 0.012s sys 0.025s
I20250905 07:47:46.523867 9971 raft_consensus.cc:357] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.524180 9971 raft_consensus.cc:383] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.524324 9971 raft_consensus.cc:738] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.524302 9926 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d13057813c0944f494e40ab24a188261" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe" is_pre_election: true
I20250905 07:47:46.524646 9971 consensus_queue.cc:260] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.524729 9926 raft_consensus.cc:2466] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.524991 9974 tablet_bootstrap.cc:492] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.525193 9974 ts_tablet_manager.cc:1397] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.045s user 0.024s sys 0.015s
I20250905 07:47:46.526018 9971 ts_tablet_manager.cc:1428] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250905 07:47:46.526288 9971 tablet_bootstrap.cc:492] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.528012 9979 raft_consensus.cc:513] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.528878 9974 raft_consensus.cc:357] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.529160 9974 raft_consensus.cc:383] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.529368 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d13057813c0944f494e40ab24a188261" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972"
I20250905 07:47:46.529629 9777 raft_consensus.cc:3058] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.530459 9971 tablet_bootstrap.cc:654] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.532311 9971 tablet_bootstrap.cc:492] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.532500 9971 ts_tablet_manager.cc:1397] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20250905 07:47:46.533061 9971 raft_consensus.cc:357] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.533223 9971 raft_consensus.cc:383] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.533313 9971 raft_consensus.cc:738] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.533584 9971 consensus_queue.cc:260] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.533912 9777 raft_consensus.cc:2466] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
I20250905 07:47:46.534168 9971 ts_tablet_manager.cc:1428] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:47:46.534541 9817 leader_election.cc:304] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.534920 9979 leader_election.cc:290] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Requested vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.529389 9974 raft_consensus.cc:738] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.535709 9974 consensus_queue.cc:260] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.538614 9971 tablet_bootstrap.cc:492] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.538869 9979 raft_consensus.cc:2802] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.538918 9972 tablet_bootstrap.cc:492] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.539146 9974 ts_tablet_manager.cc:1428] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20250905 07:47:46.539283 9972 ts_tablet_manager.cc:1397] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.057s user 0.000s sys 0.051s
I20250905 07:47:46.539369 9926 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d13057813c0944f494e40ab24a188261" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe"
I20250905 07:47:46.539615 9926 raft_consensus.cc:3058] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.540518 9972 raft_consensus.cc:357] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.540824 9971 tablet_bootstrap.cc:654] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.540863 9972 raft_consensus.cc:383] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.541498 9995 raft_consensus.cc:491] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.541766 9979 raft_consensus.cc:695] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Becoming Leader. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Running, Role: LEADER
I20250905 07:47:46.542296 9974 tablet_bootstrap.cc:492] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.542523 9972 raft_consensus.cc:738] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.542634 9979 consensus_queue.cc:237] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.542951 9978 raft_consensus.cc:491] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.543100 9978 raft_consensus.cc:513] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.544100 9972 consensus_queue.cc:260] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.545538 9972 ts_tablet_manager.cc:1428] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20250905 07:47:46.545576 9978 leader_election.cc:290] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905)
I20250905 07:47:46.542037 9995 raft_consensus.cc:513] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.546317 9972 tablet_bootstrap.cc:492] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.548115 9995 leader_election.cc:290] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.548482 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb3bc705b77c4e30a481e3b291ea2ee4" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972" is_pre_election: true
I20250905 07:47:46.550662 9926 raft_consensus.cc:2466] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
W20250905 07:47:46.551237 9817 leader_election.cc:343] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:47:46.557256 9977 raft_consensus.cc:491] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.557467 9977 raft_consensus.cc:513] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.558539 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb3bc705b77c4e30a481e3b291ea2ee4" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe" is_pre_election: true
I20250905 07:47:46.571020 9974 tablet_bootstrap.cc:654] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.569027 9977 leader_election.cc:290] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
W20250905 07:47:46.570936 9817 leader_election.cc:343] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:47:46.571633 9817 leader_election.cc:304] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e835801287e04854a5a9bbcbd764a087; no voters: 6cd0127a22db4e5b83195785b0f73972, afcca3d30c72404cbcfb08328bdb8efe
I20250905 07:47:46.573274 9995 raft_consensus.cc:2747] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:47:46.592747 9673 catalog_manager.cc:5582] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 reported cstate change: term changed from 0 to 1, leader changed from <none> to e835801287e04854a5a9bbcbd764a087 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "e835801287e04854a5a9bbcbd764a087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } }
I20250905 07:47:46.599705 9979 raft_consensus.cc:491] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.601943 9979 raft_consensus.cc:513] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.603554 9971 tablet_bootstrap.cc:492] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.605334 9971 ts_tablet_manager.cc:1397] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.069s user 0.013s sys 0.033s
I20250905 07:47:46.606843 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "90dabf9a64384373a330c56033e03b9c" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972" is_pre_election: true
I20250905 07:47:46.607084 9777 raft_consensus.cc:2466] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.607180 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "90dabf9a64384373a330c56033e03b9c" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe" is_pre_election: true
I20250905 07:47:46.607653 9817 leader_election.cc:304] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.607688 9925 raft_consensus.cc:2466] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.608330 9979 leader_election.cc:290] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.608441 9995 raft_consensus.cc:2802] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.608567 9995 raft_consensus.cc:491] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.608971 9995 raft_consensus.cc:3058] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.609194 9971 raft_consensus.cc:357] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.609380 9971 raft_consensus.cc:383] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.610126 9971 raft_consensus.cc:738] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.610399 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eda10554d4b44034b1b97c55315e5e0f" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972" is_pre_election: true
I20250905 07:47:46.610656 9777 raft_consensus.cc:2466] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afcca3d30c72404cbcfb08328bdb8efe in term 0.
I20250905 07:47:46.610713 9974 tablet_bootstrap.cc:492] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.611256 9971 consensus_queue.cc:260] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.611930 9974 ts_tablet_manager.cc:1397] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.069s user 0.006s sys 0.009s
I20250905 07:47:46.612032 9891 leader_election.cc:304] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, afcca3d30c72404cbcfb08328bdb8efe; no voters:
I20250905 07:47:46.612336 9978 raft_consensus.cc:2802] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.612462 9978 raft_consensus.cc:491] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.612555 9978 raft_consensus.cc:3058] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.612849 9974 raft_consensus.cc:357] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.613031 9974 raft_consensus.cc:383] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.613126 9974 raft_consensus.cc:738] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.613427 9974 consensus_queue.cc:260] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.616087 9972 tablet_bootstrap.cc:654] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.617486 9971 ts_tablet_manager.cc:1428] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20250905 07:47:46.617944 9971 tablet_bootstrap.cc:492] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972: Bootstrap starting.
I20250905 07:47:46.620473 9971 tablet_bootstrap.cc:654] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.621354 9995 raft_consensus.cc:513] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.623948 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "90dabf9a64384373a330c56033e03b9c" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972"
I20250905 07:47:46.624078 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "90dabf9a64384373a330c56033e03b9c" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe"
I20250905 07:47:46.624289 9978 raft_consensus.cc:513] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.624771 9995 leader_election.cc:290] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Requested vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.625787 9777 raft_consensus.cc:3058] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.626199 9925 raft_consensus.cc:3058] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.627038 9851 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d7afbcc8155649f0b144345c6c807762" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087" is_pre_election: true
I20250905 07:47:46.627648 9776 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eda10554d4b44034b1b97c55315e5e0f" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972"
W20250905 07:47:46.627774 9740 leader_election.cc:343] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:47:46.627918 9776 raft_consensus.cc:3058] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.628301 9850 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eda10554d4b44034b1b97c55315e5e0f" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087"
I20250905 07:47:46.628562 9850 raft_consensus.cc:3058] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.628562 9974 ts_tablet_manager.cc:1428] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
I20250905 07:47:46.629040 9978 leader_election.cc:290] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 election: Requested vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905)
I20250905 07:47:46.629822 9926 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d7afbcc8155649f0b144345c6c807762" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe" is_pre_election: true
I20250905 07:47:46.630275 9926 raft_consensus.cc:2466] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 0.
I20250905 07:47:46.629580 9852 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eda10554d4b44034b1b97c55315e5e0f" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087" is_pre_election: true
I20250905 07:47:46.630568 9974 tablet_bootstrap.cc:492] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.633164 9742 leader_election.cc:304] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, afcca3d30c72404cbcfb08328bdb8efe; no voters: e835801287e04854a5a9bbcbd764a087
I20250905 07:47:46.634776 9972 tablet_bootstrap.cc:492] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.634955 9972 ts_tablet_manager.cc:1397] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.089s user 0.006s sys 0.005s
I20250905 07:47:46.636123 9972 raft_consensus.cc:357] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.636595 9977 raft_consensus.cc:2802] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.637830 9977 raft_consensus.cc:491] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.637986 9977 raft_consensus.cc:3058] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.638568 9972 raft_consensus.cc:383] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.639541 9972 raft_consensus.cc:738] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.639932 9972 consensus_queue.cc:260] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.640626 9972 ts_tablet_manager.cc:1428] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20250905 07:47:46.641021 9972 tablet_bootstrap.cc:492] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.642537 9974 tablet_bootstrap.cc:654] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.643050 9972 tablet_bootstrap.cc:654] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.644624 9974 tablet_bootstrap.cc:492] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.644752 9974 ts_tablet_manager.cc:1397] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.003s
I20250905 07:47:46.645325 9850 raft_consensus.cc:2466] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afcca3d30c72404cbcfb08328bdb8efe in term 1.
I20250905 07:47:46.645581 9974 raft_consensus.cc:357] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.645711 9776 raft_consensus.cc:2466] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afcca3d30c72404cbcfb08328bdb8efe in term 1.
I20250905 07:47:46.645792 9889 leader_election.cc:304] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afcca3d30c72404cbcfb08328bdb8efe, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.645795 9974 raft_consensus.cc:383] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.645973 9974 raft_consensus.cc:738] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.646075 9978 raft_consensus.cc:2802] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.646209 9978 raft_consensus.cc:695] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 1 LEADER]: Becoming Leader. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Running, Role: LEADER
I20250905 07:47:46.646286 9974 consensus_queue.cc:260] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.646489 9978 consensus_queue.cc:237] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.648700 9925 raft_consensus.cc:2466] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
I20250905 07:47:46.649123 9817 leader_election.cc:304] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afcca3d30c72404cbcfb08328bdb8efe, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.650027 9674 catalog_manager.cc:5582] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe reported cstate change: term changed from 0 to 1, leader changed from <none> to afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67). New cstate: current_term: 1 leader_uuid: "afcca3d30c72404cbcfb08328bdb8efe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:46.650292 9971 tablet_bootstrap.cc:492] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972: No bootstrap required, opened a new log
I20250905 07:47:46.650547 9971 ts_tablet_manager.cc:1397] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.000s
I20250905 07:47:46.650977 9995 raft_consensus.cc:2802] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.651153 9995 raft_consensus.cc:695] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Becoming Leader. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Running, Role: LEADER
I20250905 07:47:46.651401 9995 consensus_queue.cc:237] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } }
I20250905 07:47:46.651597 9971 raft_consensus.cc:357] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.651840 9971 raft_consensus.cc:383] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.651933 9971 raft_consensus.cc:738] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.652236 9971 consensus_queue.cc:260] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.654501 9972 tablet_bootstrap.cc:492] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.654713 9972 ts_tablet_manager.cc:1397] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.003s
I20250905 07:47:46.654558 9674 catalog_manager.cc:5582] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 reported cstate change: term changed from 0 to 1, leader changed from <none> to e835801287e04854a5a9bbcbd764a087 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "e835801287e04854a5a9bbcbd764a087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:46.655735 9777 raft_consensus.cc:2466] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
I20250905 07:47:46.656669 9972 raft_consensus.cc:357] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.657213 9972 raft_consensus.cc:383] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.657344 9972 raft_consensus.cc:738] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.657588 9972 consensus_queue.cc:260] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.657902 9974 ts_tablet_manager.cc:1428] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.013s user 0.003s sys 0.006s
I20250905 07:47:46.658279 9972 ts_tablet_manager.cc:1428] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:47:46.658586 9972 tablet_bootstrap.cc:492] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087: Bootstrap starting.
I20250905 07:47:46.660730 9972 tablet_bootstrap.cc:654] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.661597 9974 tablet_bootstrap.cc:492] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe: Bootstrap starting.
I20250905 07:47:46.662642 9972 tablet_bootstrap.cc:492] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087: No bootstrap required, opened a new log
I20250905 07:47:46.662822 9972 ts_tablet_manager.cc:1397] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20250905 07:47:46.663489 9972 raft_consensus.cc:357] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.663664 9972 raft_consensus.cc:383] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.663762 9972 raft_consensus.cc:738] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.664011 9972 consensus_queue.cc:260] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.664295 9974 tablet_bootstrap.cc:654] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:46.664567 9972 ts_tablet_manager.cc:1428] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:47:46.667505 9971 ts_tablet_manager.cc:1428] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972: Time spent starting tablet: real 0.017s user 0.010s sys 0.008s
I20250905 07:47:46.668876 9977 raft_consensus.cc:513] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.669410 9977 leader_election.cc:290] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 election: Requested vote from peers e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.670073 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d7afbcc8155649f0b144345c6c807762" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe"
I20250905 07:47:46.670225 9925 raft_consensus.cc:3058] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.671486 9850 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d7afbcc8155649f0b144345c6c807762" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087"
I20250905 07:47:46.671694 9850 raft_consensus.cc:3058] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.673683 9974 tablet_bootstrap.cc:492] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe: No bootstrap required, opened a new log
I20250905 07:47:46.673840 9974 ts_tablet_manager.cc:1397] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20250905 07:47:46.674504 9974 raft_consensus.cc:357] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.674671 9974 raft_consensus.cc:383] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:46.674759 9974 raft_consensus.cc:738] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Initialized, Role: FOLLOWER
I20250905 07:47:46.674984 9974 consensus_queue.cc:260] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.675424 9974 ts_tablet_manager.cc:1428] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:47:46.676999 10010 raft_consensus.cc:491] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.677131 10010 raft_consensus.cc:513] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.677984 10010 leader_election.cc:290] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905)
I20250905 07:47:46.678254 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1397ac721f3e41ae99dcebddc2bba870" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972" is_pre_election: true
I20250905 07:47:46.678431 9852 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1397ac721f3e41ae99dcebddc2bba870" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087" is_pre_election: true
I20250905 07:47:46.678449 9777 raft_consensus.cc:2466] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afcca3d30c72404cbcfb08328bdb8efe in term 0.
I20250905 07:47:46.678682 9852 raft_consensus.cc:2466] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afcca3d30c72404cbcfb08328bdb8efe in term 0.
I20250905 07:47:46.678886 9891 leader_election.cc:304] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, afcca3d30c72404cbcfb08328bdb8efe; no voters:
I20250905 07:47:46.679250 10010 raft_consensus.cc:2802] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.679369 10010 raft_consensus.cc:491] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.679530 10010 raft_consensus.cc:3058] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.679884 9925 raft_consensus.cc:2466] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 1.
I20250905 07:47:46.679855 9850 raft_consensus.cc:2466] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 1.
I20250905 07:47:46.680310 9740 leader_election.cc:304] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.680639 9977 raft_consensus.cc:2802] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.680770 9977 raft_consensus.cc:695] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 1 LEADER]: Becoming Leader. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Running, Role: LEADER
I20250905 07:47:46.681085 9977 consensus_queue.cc:237] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.683359 9673 catalog_manager.cc:5582] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "6cd0127a22db4e5b83195785b0f73972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } }
I20250905 07:47:46.689931 9977 raft_consensus.cc:491] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.690102 9977 raft_consensus.cc:513] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.690743 9977 leader_election.cc:290] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.690675 10010 raft_consensus.cc:513] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.691254 9850 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb3bc705b77c4e30a481e3b291ea2ee4" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087" is_pre_election: true
I20250905 07:47:46.691542 9850 raft_consensus.cc:2466] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 0.
I20250905 07:47:46.691701 10010 leader_election.cc:290] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 election: Requested vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905)
I20250905 07:47:46.691757 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1397ac721f3e41ae99dcebddc2bba870" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972"
I20250905 07:47:46.691288 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb3bc705b77c4e30a481e3b291ea2ee4" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe" is_pre_election: true
I20250905 07:47:46.691957 9777 raft_consensus.cc:3058] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.691988 9740 leader_election.cc:304] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.692020 9925 raft_consensus.cc:2466] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 0.
I20250905 07:47:46.692305 9977 raft_consensus.cc:2802] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.692431 9977 raft_consensus.cc:491] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.692552 9977 raft_consensus.cc:3058] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.692523 9850 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1397ac721f3e41ae99dcebddc2bba870" candidate_uuid: "afcca3d30c72404cbcfb08328bdb8efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087"
I20250905 07:47:46.692737 9850 raft_consensus.cc:3058] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.701133 9850 raft_consensus.cc:2466] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afcca3d30c72404cbcfb08328bdb8efe in term 1.
I20250905 07:47:46.701134 9977 raft_consensus.cc:513] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.701658 9889 leader_election.cc:304] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afcca3d30c72404cbcfb08328bdb8efe, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.701936 10010 raft_consensus.cc:2802] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.702024 9977 leader_election.cc:290] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 election: Requested vote from peers e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.702207 10010 raft_consensus.cc:695] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 1 LEADER]: Becoming Leader. State: Replica: afcca3d30c72404cbcfb08328bdb8efe, State: Running, Role: LEADER
I20250905 07:47:46.702440 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb3bc705b77c4e30a481e3b291ea2ee4" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe"
I20250905 07:47:46.702633 9925 raft_consensus.cc:3058] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.702507 10010 consensus_queue.cc:237] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.702708 9850 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb3bc705b77c4e30a481e3b291ea2ee4" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e835801287e04854a5a9bbcbd764a087"
I20250905 07:47:46.702894 9850 raft_consensus.cc:3058] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.705864 9673 catalog_manager.cc:5582] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe reported cstate change: term changed from 0 to 1, leader changed from <none> to afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67). New cstate: current_term: 1 leader_uuid: "afcca3d30c72404cbcfb08328bdb8efe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:46.711668 9850 raft_consensus.cc:2466] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 1.
I20250905 07:47:46.711668 9777 raft_consensus.cc:2466] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afcca3d30c72404cbcfb08328bdb8efe in term 1.
I20250905 07:47:46.711670 9925 raft_consensus.cc:2466] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 1.
I20250905 07:47:46.712232 9742 leader_election.cc:304] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, afcca3d30c72404cbcfb08328bdb8efe; no voters:
I20250905 07:47:46.712471 9977 raft_consensus.cc:2802] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.712652 9977 raft_consensus.cc:695] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 1 LEADER]: Becoming Leader. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Running, Role: LEADER
I20250905 07:47:46.712921 9977 consensus_queue.cc:237] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.715145 9673 catalog_manager.cc:5582] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "6cd0127a22db4e5b83195785b0f73972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } }
I20250905 07:47:46.745569 9995 raft_consensus.cc:491] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.745770 9995 raft_consensus.cc:513] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.746434 9995 leader_election.cc:290] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.747067 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d4952e38a8e4aefa32a14640633e595" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972" is_pre_election: true
I20250905 07:47:46.747013 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d4952e38a8e4aefa32a14640633e595" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe" is_pre_election: true
I20250905 07:47:46.747274 9925 raft_consensus.cc:2466] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.747267 9777 raft_consensus.cc:2466] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.747731 9817 leader_election.cc:304] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afcca3d30c72404cbcfb08328bdb8efe, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.748008 9995 raft_consensus.cc:2802] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.748147 9995 raft_consensus.cc:491] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.748242 9995 raft_consensus.cc:3058] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.763507 9979 raft_consensus.cc:491] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:46.763710 9979 raft_consensus.cc:513] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.764412 9979 leader_election.cc:290] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.765048 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0ff44f9708145e995065016bf8d8fcd" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972" is_pre_election: true
I20250905 07:47:46.765090 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0ff44f9708145e995065016bf8d8fcd" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe" is_pre_election: true
I20250905 07:47:46.765281 9777 raft_consensus.cc:2466] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.765290 9925 raft_consensus.cc:2466] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 0.
I20250905 07:47:46.765547 9995 raft_consensus.cc:513] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.765739 9817 leader_election.cc:304] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.766024 9979 raft_consensus.cc:2802] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:46.766165 9979 raft_consensus.cc:491] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:46.766168 9995 leader_election.cc:290] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Requested vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.766278 9979 raft_consensus.cc:3058] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.766584 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d4952e38a8e4aefa32a14640633e595" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972"
I20250905 07:47:46.766754 9777 raft_consensus.cc:3058] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.767089 9925 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d4952e38a8e4aefa32a14640633e595" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe"
I20250905 07:47:46.767302 9925 raft_consensus.cc:3058] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.791625 9925 raft_consensus.cc:2466] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
I20250905 07:47:46.791610 9979 raft_consensus.cc:513] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.791599 9777 raft_consensus.cc:2466] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
I20250905 07:47:46.792243 9817 leader_election.cc:304] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afcca3d30c72404cbcfb08328bdb8efe, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.792477 9979 leader_election.cc:290] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Requested vote from peers 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65:38187), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:46.792587 9995 raft_consensus.cc:2802] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.792721 9995 raft_consensus.cc:695] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Becoming Leader. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Running, Role: LEADER
I20250905 07:47:46.793097 9777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0ff44f9708145e995065016bf8d8fcd" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd0127a22db4e5b83195785b0f73972"
I20250905 07:47:46.793175 9926 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0ff44f9708145e995065016bf8d8fcd" candidate_uuid: "e835801287e04854a5a9bbcbd764a087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe"
I20250905 07:47:46.793437 9777 raft_consensus.cc:3058] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.793370 9995 consensus_queue.cc:237] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.793488 9926 raft_consensus.cc:3058] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:46.796473 9672 catalog_manager.cc:5582] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 reported cstate change: term changed from 0 to 1, leader changed from <none> to e835801287e04854a5a9bbcbd764a087 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "e835801287e04854a5a9bbcbd764a087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } }
I20250905 07:47:46.824688 9926 raft_consensus.cc:2466] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
I20250905 07:47:46.824688 9777 raft_consensus.cc:2466] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e835801287e04854a5a9bbcbd764a087 in term 1.
I20250905 07:47:46.825273 9817 leader_election.cc:304] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afcca3d30c72404cbcfb08328bdb8efe, e835801287e04854a5a9bbcbd764a087; no voters:
I20250905 07:47:46.825574 9995 raft_consensus.cc:2802] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:46.825748 9995 raft_consensus.cc:695] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Becoming Leader. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Running, Role: LEADER
I20250905 07:47:46.826035 9995 consensus_queue.cc:237] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:46.828464 9672 catalog_manager.cc:5582] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 reported cstate change: term changed from 0 to 1, leader changed from <none> to e835801287e04854a5a9bbcbd764a087 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "e835801287e04854a5a9bbcbd764a087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } }
I20250905 07:47:46.906908 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:46.909701 10021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:46.910125 10022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:46.911078 10024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:46.911863 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:46.912135 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:46.912220 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:46.912278 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058466912259 us; error 0 us; skew 500 ppm
I20250905 07:47:46.912446 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:46.913769 9109 webserver.cc:480] Webserver started at http://127.8.229.68:42165/ using document root <none> and password file <none>
I20250905 07:47:46.914002 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:46.914094 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:46.914248 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:46.914892 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "d9ad88ff151447fa925f4a5ef84ab63d"
format_stamp: "Formatted at 2025-09-05 07:47:46 on dist-test-slave-pgzt"
I20250905 07:47:46.917466 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:47:46.919137 10029 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:46.919520 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:47:46.919647 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "d9ad88ff151447fa925f4a5ef84ab63d"
format_stamp: "Formatted at 2025-09-05 07:47:46 on dist-test-slave-pgzt"
I20250905 07:47:46.919770 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:46.931924 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:46.932497 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:46.933087 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:46.933969 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:46.934058 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:46.934160 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:46.934243 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:46.953953 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:39503
I20250905 07:47:46.953989 10091 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:39503 every 8 connection(s)
I20250905 07:47:46.962101 10092 heartbeater.cc:344] Connected to a master server at 127.8.229.126:36343
I20250905 07:47:46.962239 10092 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:46.962687 10092 heartbeater.cc:507] Master 127.8.229.126:36343 requested a full tablet report, sending...
I20250905 07:47:46.963650 9672 ts_manager.cc:194] Registered new tserver with Master: d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68:39503)
I20250905 07:47:46.964457 9672 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57350
I20250905 07:47:47.011425 9728 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:47:47.014990 9995 consensus_queue.cc:1035] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.021526 9995 consensus_queue.cc:1035] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250905 07:47:47.025353 9850 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "d13057813c0944f494e40ab24a188261"
dest_uuid: "e835801287e04854a5a9bbcbd764a087"
mode: GRACEFUL
new_leader_uuid: "6cd0127a22db4e5b83195785b0f73972"
from {username='slave'} at 127.0.0.1:52214
I20250905 07:47:47.025645 9850 raft_consensus.cc:604] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Received request to transfer leadership to TS 6cd0127a22db4e5b83195785b0f73972
I20250905 07:47:47.026294 9728 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:47:47.026614 9728 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:47:47.085006 9995 consensus_queue.cc:1035] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.089660 9995 consensus_queue.cc:1035] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.089708 10010 consensus_queue.cc:1035] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.094184 10005 consensus_queue.cc:1035] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.112084 9977 consensus_queue.cc:1035] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.116242 10013 consensus_queue.cc:1035] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.137195 10010 consensus_queue.cc:1035] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.141317 10005 consensus_queue.cc:1035] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.158404 10013 consensus_queue.cc:1035] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.163941 10107 consensus_queue.cc:1035] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.240026 9995 consensus_queue.cc:1035] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.244100 9996 consensus_queue.cc:1035] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.356680 9995 consensus_queue.cc:1035] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.361398 9996 consensus_queue.cc:1035] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:47.456221 9995 raft_consensus.cc:991] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087: : Instructing follower 6cd0127a22db4e5b83195785b0f73972 to start an election
I20250905 07:47:47.456439 9995 raft_consensus.cc:1079] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Signalling peer 6cd0127a22db4e5b83195785b0f73972 to start an election
I20250905 07:47:47.457319 9777 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "d13057813c0944f494e40ab24a188261"
dest_uuid: "6cd0127a22db4e5b83195785b0f73972"
from {username='slave'} at 127.0.0.1:59276
I20250905 07:47:47.457548 9777 raft_consensus.cc:491] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:47:47.457685 9777 raft_consensus.cc:3058] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:47:47.548861 9777 raft_consensus.cc:513] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:47.549654 9777 leader_election.cc:290] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 2 election: Requested vote from peers e835801287e04854a5a9bbcbd764a087 (127.8.229.66:44905), afcca3d30c72404cbcfb08328bdb8efe (127.8.229.67:36149)
I20250905 07:47:47.550189 9850 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d13057813c0944f494e40ab24a188261" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e835801287e04854a5a9bbcbd764a087"
I20250905 07:47:47.550339 9926 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d13057813c0944f494e40ab24a188261" candidate_uuid: "6cd0127a22db4e5b83195785b0f73972" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "afcca3d30c72404cbcfb08328bdb8efe"
I20250905 07:47:47.550450 9850 raft_consensus.cc:3053] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:47:47.550529 9926 raft_consensus.cc:3058] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:47:47.550577 9850 raft_consensus.cc:738] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e835801287e04854a5a9bbcbd764a087, State: Running, Role: LEADER
I20250905 07:47:47.550926 9850 consensus_queue.cc:260] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:47.551369 9850 raft_consensus.cc:3058] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:47:47.552891 9926 raft_consensus.cc:2466] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 2.
I20250905 07:47:47.553388 9742 leader_election.cc:304] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd0127a22db4e5b83195785b0f73972, afcca3d30c72404cbcfb08328bdb8efe; no voters:
I20250905 07:47:47.553691 10107 raft_consensus.cc:2802] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:47:47.553862 10107 raft_consensus.cc:695] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 2 LEADER]: Becoming Leader. State: Replica: 6cd0127a22db4e5b83195785b0f73972, State: Running, Role: LEADER
I20250905 07:47:47.554131 10107 consensus_queue.cc:237] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } }
I20250905 07:47:47.556234 9672 catalog_manager.cc:5582] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 reported cstate change: term changed from 1 to 2, leader changed from e835801287e04854a5a9bbcbd764a087 (127.8.229.66) to 6cd0127a22db4e5b83195785b0f73972 (127.8.229.65). New cstate: current_term: 2 leader_uuid: "6cd0127a22db4e5b83195785b0f73972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } }
I20250905 07:47:47.645325 9850 raft_consensus.cc:2466] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cd0127a22db4e5b83195785b0f73972 in term 2.
I20250905 07:47:47.965637 10092 heartbeater.cc:499] Master 127.8.229.126:36343 was elected leader, sending a full tablet report...
I20250905 07:47:47.988955 9850 raft_consensus.cc:1273] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 2 FOLLOWER]: Refusing update from remote peer 6cd0127a22db4e5b83195785b0f73972: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:47:47.989574 10013 consensus_queue.cc:1035] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:47.994436 9926 raft_consensus.cc:1273] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 2 FOLLOWER]: Refusing update from remote peer 6cd0127a22db4e5b83195785b0f73972: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:47:47.995607 10107 consensus_queue.cc:1035] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:49.026965 9728 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:47:49.028371 9728 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:47:49.028513 9728 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:47:51.028987 9728 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:47:51.030352 9728 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:47:51.030544 9728 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:47:52.027634 9926 consensus_queue.cc:237] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } attrs { replace: true } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } }
I20250905 07:47:52.030691 9851 raft_consensus.cc:1273] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Refusing update from remote peer afcca3d30c72404cbcfb08328bdb8efe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.030691 9777 raft_consensus.cc:1273] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Refusing update from remote peer afcca3d30c72404cbcfb08328bdb8efe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.031360 10128 consensus_queue.cc:1035] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:52.031751 10010 consensus_queue.cc:1035] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:52.034921 10010 raft_consensus.cc:2953] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } attrs { replace: true } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.035722 9851 raft_consensus.cc:2953] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } attrs { replace: true } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.036768 9777 raft_consensus.cc:2953] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } attrs { replace: true } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
W20250905 07:47:52.038110 9891 consensus_peers.cc:489] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe -> Peer d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68:39503): Couldn't send request to peer d9ad88ff151447fa925f4a5ef84ab63d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1397ac721f3e41ae99dcebddc2bba870. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:47:52.041549 9674 catalog_manager.cc:5582] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 reported cstate change: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "afcca3d30c72404cbcfb08328bdb8efe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } attrs { replace: true } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.049348 9777 consensus_queue.cc:237] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } attrs { replace: true } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } }
I20250905 07:47:52.051920 9851 raft_consensus.cc:1273] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Refusing update from remote peer 6cd0127a22db4e5b83195785b0f73972: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.055656 9977 consensus_queue.cc:1035] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:52.057202 9926 raft_consensus.cc:1273] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Refusing update from remote peer 6cd0127a22db4e5b83195785b0f73972: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.058151 10140 consensus_queue.cc:1035] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:52.059738 9977 raft_consensus.cc:2953] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } attrs { replace: true } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.060331 9851 raft_consensus.cc:2953] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } attrs { replace: true } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
W20250905 07:47:52.063467 9742 consensus_peers.cc:489] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 -> Peer d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68:39503): Couldn't send request to peer d9ad88ff151447fa925f4a5ef84ab63d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7afbcc8155649f0b144345c6c807762. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:47:52.066260 9672 catalog_manager.cc:5582] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 reported cstate change: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "6cd0127a22db4e5b83195785b0f73972" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } attrs { replace: true } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.067633 9925 consensus_queue.cc:237] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } }
W20250905 07:47:52.072492 9891 consensus_peers.cc:489] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe -> Peer d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68:39503): Couldn't send request to peer d9ad88ff151447fa925f4a5ef84ab63d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eda10554d4b44034b1b97c55315e5e0f. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:47:52.072885 9851 raft_consensus.cc:1273] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Refusing update from remote peer afcca3d30c72404cbcfb08328bdb8efe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.073154 9777 raft_consensus.cc:1273] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Refusing update from remote peer afcca3d30c72404cbcfb08328bdb8efe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.074143 10010 consensus_queue.cc:1035] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:52.074611 10010 consensus_queue.cc:1035] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:52.075302 9925 raft_consensus.cc:2953] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } attrs { replace: true } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.081157 10010 raft_consensus.cc:2953] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.081770 9851 raft_consensus.cc:2953] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.086019 9672 catalog_manager.cc:5582] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 reported cstate change: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "afcca3d30c72404cbcfb08328bdb8efe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.095255 9777 raft_consensus.cc:2953] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.095508 9851 consensus_queue.cc:237] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } }
I20250905 07:47:52.098312 9776 raft_consensus.cc:1273] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Refusing update from remote peer e835801287e04854a5a9bbcbd764a087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.099213 10124 consensus_queue.cc:1035] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250905 07:47:52.107201 9817 consensus_peers.cc:489] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 -> Peer d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68:39503): Couldn't send request to peer d9ad88ff151447fa925f4a5ef84ab63d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d4952e38a8e4aefa32a14640633e595. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:47:52.106763 10124 raft_consensus.cc:2953] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.107699 9925 raft_consensus.cc:1273] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Refusing update from remote peer e835801287e04854a5a9bbcbd764a087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.108316 10155 consensus_queue.cc:1035] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:47:52.108261 9776 raft_consensus.cc:2953] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.111217 9925 raft_consensus.cc:2953] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } } }
I20250905 07:47:52.112188 9672 catalog_manager.cc:5582] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 reported cstate change: config changed from index -1 to 2, NON_VOTER d9ad88ff151447fa925f4a5ef84ab63d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "e835801287e04854a5a9bbcbd764a087" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6cd0127a22db4e5b83195785b0f73972" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38187 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e835801287e04854a5a9bbcbd764a087" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afcca3d30c72404cbcfb08328bdb8efe" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 36149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9ad88ff151447fa925f4a5ef84ab63d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 39503 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:47:52.212693 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
W20250905 07:47:52.216735 9660 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20250905 07:47:52.218863 9723 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250905 07:47:52.219058 9723 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250905 07:47:52.222982 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:52.223311 9109 tablet_replica.cc:331] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.223548 9109 raft_consensus.cc:2241] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.223905 9109 raft_consensus.cc:2270] T eb3bc705b77c4e30a481e3b291ea2ee4 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.224555 9109 tablet_replica.cc:331] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.224726 9109 raft_consensus.cc:2241] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.224890 9109 raft_consensus.cc:2270] T c0ff44f9708145e995065016bf8d8fcd P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.225484 9109 tablet_replica.cc:331] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.225649 9109 raft_consensus.cc:2241] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.226018 9109 raft_consensus.cc:2270] T d7afbcc8155649f0b144345c6c807762 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.226665 9109 tablet_replica.cc:331] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.226912 9109 raft_consensus.cc:2241] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.227079 9109 raft_consensus.cc:2270] T 90dabf9a64384373a330c56033e03b9c P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.227631 9109 tablet_replica.cc:331] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.227793 9109 raft_consensus.cc:2241] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.227998 9109 raft_consensus.cc:2270] T 3d4952e38a8e4aefa32a14640633e595 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.228691 9109 tablet_replica.cc:331] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.228865 9109 raft_consensus.cc:2241] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.229164 9109 raft_consensus.cc:2270] T d13057813c0944f494e40ab24a188261 P 6cd0127a22db4e5b83195785b0f73972 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.229594 9109 tablet_replica.cc:331] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.229743 9109 raft_consensus.cc:2241] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.229921 9109 raft_consensus.cc:2270] T eda10554d4b44034b1b97c55315e5e0f P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.230525 9109 tablet_replica.cc:331] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972: stopping tablet replica
I20250905 07:47:52.230707 9109 raft_consensus.cc:2241] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.230901 9109 raft_consensus.cc:2270] T 1397ac721f3e41ae99dcebddc2bba870 P 6cd0127a22db4e5b83195785b0f73972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.238166 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:47:52.243291 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
W20250905 07:47:52.246084 9723 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250905 07:47:52.246572 9723 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250905 07:47:52.253516 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:52.253762 9109 tablet_replica.cc:331] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.253985 9109 raft_consensus.cc:2241] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.254195 9109 raft_consensus.cc:2270] T d7afbcc8155649f0b144345c6c807762 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.254999 9109 tablet_replica.cc:331] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.255162 9109 raft_consensus.cc:2241] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.255559 9109 raft_consensus.cc:2270] T 3d4952e38a8e4aefa32a14640633e595 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.256235 9109 tablet_replica.cc:331] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.256419 9109 raft_consensus.cc:2241] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.256661 9109 raft_consensus.cc:2270] T 90dabf9a64384373a330c56033e03b9c P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.257112 9109 tablet_replica.cc:331] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.257262 9109 raft_consensus.cc:2241] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.257515 9109 raft_consensus.cc:2270] T c0ff44f9708145e995065016bf8d8fcd P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.257990 9109 tablet_replica.cc:331] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.258180 9109 raft_consensus.cc:2241] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.258379 9109 raft_consensus.cc:2270] T d13057813c0944f494e40ab24a188261 P e835801287e04854a5a9bbcbd764a087 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.258805 9109 tablet_replica.cc:331] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.258960 9109 raft_consensus.cc:2241] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.259135 9109 raft_consensus.cc:2270] T eb3bc705b77c4e30a481e3b291ea2ee4 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.259790 9109 tablet_replica.cc:331] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.259982 9109 raft_consensus.cc:2241] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.260162 9109 raft_consensus.cc:2270] T eda10554d4b44034b1b97c55315e5e0f P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.260828 9109 tablet_replica.cc:331] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087: stopping tablet replica
I20250905 07:47:52.260969 9109 raft_consensus.cc:2241] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.261162 9109 raft_consensus.cc:2270] T 1397ac721f3e41ae99dcebddc2bba870 P e835801287e04854a5a9bbcbd764a087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.267565 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:47:52.272066 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
W20250905 07:47:52.275969 9723 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250905 07:47:52.276072 9723 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250905 07:47:52.279081 9723 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250905 07:47:52.279273 9723 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250905 07:47:52.282160 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:52.282445 9109 tablet_replica.cc:331] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.282593 9109 raft_consensus.cc:2241] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.282768 9109 raft_consensus.cc:2270] T c0ff44f9708145e995065016bf8d8fcd P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.283306 9109 tablet_replica.cc:331] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.283483 9109 raft_consensus.cc:2241] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.283789 9109 raft_consensus.cc:2270] T 1397ac721f3e41ae99dcebddc2bba870 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.284451 9109 tablet_replica.cc:331] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.284572 9109 raft_consensus.cc:2241] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.284713 9109 raft_consensus.cc:2270] T 90dabf9a64384373a330c56033e03b9c P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.285099 9109 tablet_replica.cc:331] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.285238 9109 raft_consensus.cc:2241] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.285368 9109 raft_consensus.cc:2270] T d7afbcc8155649f0b144345c6c807762 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.285882 9109 tablet_replica.cc:331] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.285990 9109 raft_consensus.cc:2241] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.286139 9109 raft_consensus.cc:2270] T 3d4952e38a8e4aefa32a14640633e595 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.286656 9109 tablet_replica.cc:331] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.286777 9109 raft_consensus.cc:2241] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.286888 9109 raft_consensus.cc:2270] T d13057813c0944f494e40ab24a188261 P afcca3d30c72404cbcfb08328bdb8efe [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.287258 9109 tablet_replica.cc:331] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.287362 9109 raft_consensus.cc:2241] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.287715 9109 raft_consensus.cc:2270] T eda10554d4b44034b1b97c55315e5e0f P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.288282 9109 tablet_replica.cc:331] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe: stopping tablet replica
I20250905 07:47:52.288403 9109 raft_consensus.cc:2241] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:52.288525 9109 raft_consensus.cc:2270] T eb3bc705b77c4e30a481e3b291ea2ee4 P afcca3d30c72404cbcfb08328bdb8efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.303793 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:47:52.307802 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:47:52.316234 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:52.329053 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:47:52.331761 9109 master.cc:561] Master@127.8.229.126:36343 shutting down...
I20250905 07:47:52.338982 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:52.339215 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:52.339340 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P ba6f3153563043fe860edb6e045f4055: stopping tablet replica
I20250905 07:47:52.343233 9109 master.cc:583] Master@127.8.229.126:36343 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn (7423 ms)
[ RUN ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing
I20250905 07:47:52.360646 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:40195,127.8.229.125:38257,127.8.229.124:34865
I20250905 07:47:52.361235 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:52.363830 10159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.363921 10158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.364176 10161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.364995 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:52.365269 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:52.365341 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:52.365398 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058472365371 us; error 0 us; skew 500 ppm
I20250905 07:47:52.365525 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:52.366664 9109 webserver.cc:480] Webserver started at http://127.8.229.126:45911/ using document root <none> and password file <none>
I20250905 07:47:52.366864 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:52.366940 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:52.367046 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:52.367710 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "386ac99d10cd43849a941982ee5c4e81"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.370150 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:47:52.371834 10166 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.372208 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:47:52.372323 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "386ac99d10cd43849a941982ee5c4e81"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.372483 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:52.388720 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:52.389364 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:52.402302 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:40195
I20250905 07:47:52.402338 10217 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:40195 every 8 connection(s)
I20250905 07:47:52.403723 10218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:52.403736 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:52.406333 10220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.406893 10218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } has no permanent_uuid. Determining permanent_uuid...
W20250905 07:47:52.406854 10221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.408619 10223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.409552 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:52.409835 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:52.409914 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:52.410004 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058472409953 us; error 0 us; skew 500 ppm
I20250905 07:47:52.410145 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:52.412541 9109 webserver.cc:480] Webserver started at http://127.8.229.125:45325/ using document root <none> and password file <none>
I20250905 07:47:52.412775 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:52.412878 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:52.413003 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:52.413820 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root/instance:
uuid: "9379a68a9e76490db703b3c0f2b6caf9"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.415349 10218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:52.416910 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
W20250905 07:47:52.417295 10218 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.8.229.125:38257: Network error: Client connection negotiation failed: client connection to 127.8.229.125:38257: connect: Connection refused (error 111)
I20250905 07:47:52.418593 10231 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.418920 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:47:52.419044 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root
uuid: "9379a68a9e76490db703b3c0f2b6caf9"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.419178 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:52.429940 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:52.430521 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:52.442930 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.125:38257
I20250905 07:47:52.442973 10282 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.125:38257 every 8 connection(s)
I20250905 07:47:52.444309 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:47:52.444475 10283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250905 07:47:52.446846 10285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.446935 10286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.448592 10288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.448679 10283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:52.448961 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:52.449260 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:52.449369 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:52.449465 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058472449421 us; error 0 us; skew 500 ppm
I20250905 07:47:52.449689 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:52.451035 9109 webserver.cc:480] Webserver started at http://127.8.229.124:41769/ using document root <none> and password file <none>
I20250905 07:47:52.451285 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:52.451397 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:52.451553 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:52.452137 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root/instance:
uuid: "b338d58c7fc64e4c9728743867a30844"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.455039 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20250905 07:47:52.455497 10283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:52.457152 10295 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.457584 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:47:52.457710 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root
uuid: "b338d58c7fc64e4c9728743867a30844"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.457881 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:52.462798 10283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } has no permanent_uuid. Determining permanent_uuid...
W20250905 07:47:52.464613 10283 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.8.229.124:34865: Network error: Client connection negotiation failed: client connection to 127.8.229.124:34865: connect: Connection refused (error 111)
I20250905 07:47:52.470095 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:52.470523 10218 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } attempt: 1
I20250905 07:47:52.470685 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:52.477336 10218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } has no permanent_uuid. Determining permanent_uuid...
W20250905 07:47:52.479259 10218 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.8.229.124:34865: Network error: Client connection negotiation failed: client connection to 127.8.229.124:34865: connect: Connection refused (error 111)
I20250905 07:47:52.485826 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.124:34865
I20250905 07:47:52.485880 10347 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.124:34865 every 8 connection(s)
I20250905 07:47:52.486743 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:47:52.487093 10348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:52.489372 10348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:52.495054 10348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:52.499914 10348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } has no permanent_uuid. Determining permanent_uuid...
I20250905 07:47:52.500485 10218 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } attempt: 1
I20250905 07:47:52.508847 10348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: Bootstrap starting.
I20250905 07:47:52.510351 10218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81: Bootstrap starting.
I20250905 07:47:52.510725 10348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:52.512161 10218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:52.512903 10348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: No bootstrap required, opened a new log
I20250905 07:47:52.513743 10348 raft_consensus.cc:357] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.513965 10348 raft_consensus.cc:383] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:52.514034 10218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81: No bootstrap required, opened a new log
I20250905 07:47:52.514073 10348 raft_consensus.cc:738] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b338d58c7fc64e4c9728743867a30844, State: Initialized, Role: FOLLOWER
I20250905 07:47:52.514379 10348 consensus_queue.cc:260] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.514870 10283 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } attempt: 1
I20250905 07:47:52.514910 10218 raft_consensus.cc:357] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.515101 10218 raft_consensus.cc:383] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:52.515190 10218 raft_consensus.cc:738] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386ac99d10cd43849a941982ee5c4e81, State: Initialized, Role: FOLLOWER
I20250905 07:47:52.515062 10355 sys_catalog.cc:455] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.515298 10355 sys_catalog.cc:458] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:52.515326 10348 sys_catalog.cc:564] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:47:52.515539 10218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.516353 10356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.516532 10356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:52.516660 10218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:47:52.524271 10283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: Bootstrap starting.
I20250905 07:47:52.525111 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250905 07:47:52.526376 10283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: Neither blocks nor log segments found. Creating new log.
W20250905 07:47:52.526877 10369 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250905 07:47:52.527004 10369 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250905 07:47:52.528558 10283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: No bootstrap required, opened a new log
I20250905 07:47:52.529187 10283 raft_consensus.cc:357] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.529366 10283 raft_consensus.cc:383] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:52.529443 10283 raft_consensus.cc:738] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9379a68a9e76490db703b3c0f2b6caf9, State: Initialized, Role: FOLLOWER
I20250905 07:47:52.529696 10283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.530347 10379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.530553 10379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:52.530707 10283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [sys.catalog]: configured and running, proceeding with master startup.
W20250905 07:47:52.533560 10381 catalog_manager.cc:1560] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250905 07:47:52.533668 10381 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250905 07:47:52.534976 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250905 07:47:52.535859 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:52.536130 10391 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250905 07:47:52.536227 10391 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250905 07:47:52.538708 10392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.539042 10393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.539757 9109 server_base.cc:1047] running on GCE node
W20250905 07:47:52.539774 10395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.540144 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:52.540235 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:52.540303 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058472540283 us; error 0 us; skew 500 ppm
I20250905 07:47:52.540478 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:52.541623 9109 webserver.cc:480] Webserver started at http://127.8.229.65:35955/ using document root <none> and password file <none>
I20250905 07:47:52.541847 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:52.541929 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:52.542083 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:52.542701 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "e021f9e358ae427c86dc337066282a1d"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.545055 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:47:52.546590 10400 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.546904 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:47:52.547014 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "e021f9e358ae427c86dc337066282a1d"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.547118 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:52.565044 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:52.565654 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:52.566223 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:52.567144 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:52.567242 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.567361 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:52.567454 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.582454 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:36149
I20250905 07:47:52.582499 10462 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:36149 every 8 connection(s)
I20250905 07:47:52.585306 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:47:52.597725 10463 heartbeater.cc:344] Connected to a master server at 127.8.229.124:34865
I20250905 07:47:52.597920 10463 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.598496 10463 heartbeater.cc:507] Master 127.8.229.124:34865 requested a full tablet report, sending...
I20250905 07:47:52.600849 10313 ts_manager.cc:194] Registered new tserver with Master: e021f9e358ae427c86dc337066282a1d (127.8.229.65:36149)
W20250905 07:47:52.602267 10470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.603329 10471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.603950 10465 heartbeater.cc:344] Connected to a master server at 127.8.229.125:38257
I20250905 07:47:52.604131 10465 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.604684 10465 heartbeater.cc:507] Master 127.8.229.125:38257 requested a full tablet report, sending...
I20250905 07:47:52.606104 10248 ts_manager.cc:194] Registered new tserver with Master: e021f9e358ae427c86dc337066282a1d (127.8.229.65:36149)
I20250905 07:47:52.607776 10464 heartbeater.cc:344] Connected to a master server at 127.8.229.126:40195
W20250905 07:47:52.607936 10473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.608095 10464 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.608398 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:52.608691 10464 heartbeater.cc:507] Master 127.8.229.126:40195 requested a full tablet report, sending...
I20250905 07:47:52.608711 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:52.608839 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:52.608917 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058472608879 us; error 0 us; skew 500 ppm
I20250905 07:47:52.609128 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:52.609764 10183 ts_manager.cc:194] Registered new tserver with Master: e021f9e358ae427c86dc337066282a1d (127.8.229.65:36149)
I20250905 07:47:52.610435 9109 webserver.cc:480] Webserver started at http://127.8.229.66:35307/ using document root <none> and password file <none>
I20250905 07:47:52.610649 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:52.610745 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:52.610873 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:52.611558 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "2de3982129844172aef574069c768423"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.614012 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20250905 07:47:52.615602 10478 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.615947 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:47:52.616055 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "2de3982129844172aef574069c768423"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.616197 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:52.626754 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:52.627362 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:52.627995 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:52.628865 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:52.628927 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.629009 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:52.629055 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.644064 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:36789
I20250905 07:47:52.644102 10540 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:36789 every 8 connection(s)
I20250905 07:47:52.646579 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:52.650658 10547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.654127 10548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:52.659869 10551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:52.660712 10543 heartbeater.cc:344] Connected to a master server at 127.8.229.125:38257
I20250905 07:47:52.660887 10543 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.661441 10543 heartbeater.cc:507] Master 127.8.229.125:38257 requested a full tablet report, sending...
I20250905 07:47:52.662068 10542 heartbeater.cc:344] Connected to a master server at 127.8.229.126:40195
I20250905 07:47:52.662222 10542 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.662611 10542 heartbeater.cc:507] Master 127.8.229.126:40195 requested a full tablet report, sending...
I20250905 07:47:52.663514 10248 ts_manager.cc:194] Registered new tserver with Master: 2de3982129844172aef574069c768423 (127.8.229.66:36789)
I20250905 07:47:52.664625 10541 heartbeater.cc:344] Connected to a master server at 127.8.229.124:34865
I20250905 07:47:52.664763 10541 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.664888 10183 ts_manager.cc:194] Registered new tserver with Master: 2de3982129844172aef574069c768423 (127.8.229.66:36789)
I20250905 07:47:52.665186 10541 heartbeater.cc:507] Master 127.8.229.124:34865 requested a full tablet report, sending...
I20250905 07:47:52.665953 10313 ts_manager.cc:194] Registered new tserver with Master: 2de3982129844172aef574069c768423 (127.8.229.66:36789)
I20250905 07:47:52.666352 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:52.666605 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:52.666688 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:52.666750 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058472666731 us; error 0 us; skew 500 ppm
I20250905 07:47:52.666914 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:52.668087 9109 webserver.cc:480] Webserver started at http://127.8.229.67:43279/ using document root <none> and password file <none>
I20250905 07:47:52.668314 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:52.668403 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:52.668548 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:52.669168 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.671658 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:47:52.673267 10556 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.673698 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:47:52.673791 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9"
format_stamp: "Formatted at 2025-09-05 07:47:52 on dist-test-slave-pgzt"
I20250905 07:47:52.673888 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:52.699642 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:52.700238 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:52.700817 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:52.701824 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:52.701906 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.702024 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:52.702090 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:52.714594 10356 raft_consensus.cc:491] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:52.714890 10356 raft_consensus.cc:513] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.715802 10356 leader_election.cc:290] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9379a68a9e76490db703b3c0f2b6caf9 (127.8.229.125:38257), b338d58c7fc64e4c9728743867a30844 (127.8.229.124:34865)
I20250905 07:47:52.716486 10258 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "386ac99d10cd43849a941982ee5c4e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9379a68a9e76490db703b3c0f2b6caf9" is_pre_election: true
I20250905 07:47:52.716642 10323 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "386ac99d10cd43849a941982ee5c4e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b338d58c7fc64e4c9728743867a30844" is_pre_election: true
I20250905 07:47:52.716856 10258 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 386ac99d10cd43849a941982ee5c4e81 in term 0.
I20250905 07:47:52.716914 10323 raft_consensus.cc:2466] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 386ac99d10cd43849a941982ee5c4e81 in term 0.
I20250905 07:47:52.717617 10169 leader_election.cc:304] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 386ac99d10cd43849a941982ee5c4e81, 9379a68a9e76490db703b3c0f2b6caf9; no voters:
I20250905 07:47:52.718004 10356 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:47:52.718173 10356 raft_consensus.cc:491] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:52.718362 10356 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:52.721508 10356 raft_consensus.cc:513] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.721832 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:42695
I20250905 07:47:52.721877 10618 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:42695 every 8 connection(s)
I20250905 07:47:52.722450 10356 leader_election.cc:290] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [CANDIDATE]: Term 1 election: Requested vote from peers 9379a68a9e76490db703b3c0f2b6caf9 (127.8.229.125:38257), b338d58c7fc64e4c9728743867a30844 (127.8.229.124:34865)
I20250905 07:47:52.722558 10258 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "386ac99d10cd43849a941982ee5c4e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9379a68a9e76490db703b3c0f2b6caf9"
I20250905 07:47:52.722807 10258 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:52.723317 10323 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "386ac99d10cd43849a941982ee5c4e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b338d58c7fc64e4c9728743867a30844"
I20250905 07:47:52.723531 10323 raft_consensus.cc:3058] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:52.726838 10258 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 386ac99d10cd43849a941982ee5c4e81 in term 1.
I20250905 07:47:52.727368 10169 leader_election.cc:304] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 386ac99d10cd43849a941982ee5c4e81, 9379a68a9e76490db703b3c0f2b6caf9; no voters:
I20250905 07:47:52.727834 10356 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:52.728365 10356 raft_consensus.cc:695] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 1 LEADER]: Becoming Leader. State: Replica: 386ac99d10cd43849a941982ee5c4e81, State: Running, Role: LEADER
I20250905 07:47:52.728762 10356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:52.727380 10323 raft_consensus.cc:2466] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 386ac99d10cd43849a941982ee5c4e81 in term 1.
I20250905 07:47:52.730655 10623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386ac99d10cd43849a941982ee5c4e81. Latest consensus state: current_term: 1 leader_uuid: "386ac99d10cd43849a941982ee5c4e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.734202 10623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:52.741937 10619 heartbeater.cc:344] Connected to a master server at 127.8.229.124:34865
I20250905 07:47:52.742106 10619 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.742775 10619 heartbeater.cc:507] Master 127.8.229.124:34865 requested a full tablet report, sending...
I20250905 07:47:52.744065 10313 ts_manager.cc:194] Registered new tserver with Master: 1ea60c6ba39d4b3a835e8a53732e7bd9 (127.8.229.67:42695)
I20250905 07:47:52.748996 10628 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:47:52.750229 10620 heartbeater.cc:344] Connected to a master server at 127.8.229.126:40195
I20250905 07:47:52.750420 10620 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.750929 10620 heartbeater.cc:507] Master 127.8.229.126:40195 requested a full tablet report, sending...
I20250905 07:47:52.752074 10183 ts_manager.cc:194] Registered new tserver with Master: 1ea60c6ba39d4b3a835e8a53732e7bd9 (127.8.229.67:42695)
I20250905 07:47:52.752362 10628 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:47:52.752812 10621 heartbeater.cc:344] Connected to a master server at 127.8.229.125:38257
I20250905 07:47:52.752993 10621 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:52.753656 10621 heartbeater.cc:507] Master 127.8.229.125:38257 requested a full tablet report, sending...
I20250905 07:47:52.754675 10248 ts_manager.cc:194] Registered new tserver with Master: 1ea60c6ba39d4b3a835e8a53732e7bd9 (127.8.229.67:42695)
I20250905 07:47:52.755375 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.001285803s
I20250905 07:47:52.755993 10258 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Refusing update from remote peer 386ac99d10cd43849a941982ee5c4e81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.756083 10323 raft_consensus.cc:1273] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Refusing update from remote peer 386ac99d10cd43849a941982ee5c4e81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:47:52.756604 10356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:52.756884 10623 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:47:52.759817 10379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386ac99d10cd43849a941982ee5c4e81. Latest consensus state: current_term: 1 leader_uuid: "386ac99d10cd43849a941982ee5c4e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.760129 10379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:52.760874 10355 sys_catalog.cc:455] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386ac99d10cd43849a941982ee5c4e81. Latest consensus state: current_term: 1 leader_uuid: "386ac99d10cd43849a941982ee5c4e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.761152 10355 sys_catalog.cc:458] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:52.761482 10623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "386ac99d10cd43849a941982ee5c4e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.761683 10623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:52.762648 10379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "386ac99d10cd43849a941982ee5c4e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.762833 10379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:52.762703 10356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "386ac99d10cd43849a941982ee5c4e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.762988 10356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:52.764140 10355 sys_catalog.cc:455] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "386ac99d10cd43849a941982ee5c4e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:52.764394 10355 sys_catalog.cc:458] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: This master's current role is: FOLLOWER
I20250905 07:47:52.765290 10628 catalog_manager.cc:1349] Generated new cluster ID: ba4ef1442d484da78f1062b53cb82a9c
I20250905 07:47:52.765367 10628 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:47:52.789847 10628 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:47:52.790983 10628 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:47:52.811323 10628 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81: Generated new TSK 0
I20250905 07:47:52.811684 10628 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:47:53.534974 10381 catalog_manager.cc:1261] Loaded cluster ID: ba4ef1442d484da78f1062b53cb82a9c
I20250905 07:47:53.535089 10381 catalog_manager.cc:1554] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: loading cluster ID for follower catalog manager: success
I20250905 07:47:53.537068 10381 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: acquiring CA information for follower catalog manager: success
I20250905 07:47:53.537385 10391 catalog_manager.cc:1261] Loaded cluster ID: ba4ef1442d484da78f1062b53cb82a9c
I20250905 07:47:53.537468 10391 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: loading cluster ID for follower catalog manager: success
I20250905 07:47:53.538164 10381 catalog_manager.cc:1604] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250905 07:47:53.539319 10391 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: acquiring CA information for follower catalog manager: success
I20250905 07:47:53.540297 10391 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250905 07:47:53.612159 10183 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57938
I20250905 07:47:53.667265 10183 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57946
I20250905 07:47:53.754611 10183 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57958
I20250905 07:47:54.525035 10368 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:47:54.613385 10464 heartbeater.cc:499] Master 127.8.229.126:40195 was elected leader, sending a full tablet report...
I20250905 07:47:54.668466 10542 heartbeater.cc:499] Master 127.8.229.126:40195 was elected leader, sending a full tablet report...
I20250905 07:47:54.755870 10620 heartbeater.cc:499] Master 127.8.229.126:40195 was elected leader, sending a full tablet report...
I20250905 07:47:54.911629 9109 test_util.cc:276] Using random seed: 239019876
I20250905 07:47:54.932646 10183 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57972:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250905 07:47:54.970178 10506 tablet_service.cc:1468] Processing CreateTablet for tablet 49d138ad00434a6ea2a10942ff2dec27 (DEFAULT_TABLE table=test-workload [id=8debc1751fb84af0a854fd82b5989ba4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20250905 07:47:54.970783 10506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49d138ad00434a6ea2a10942ff2dec27. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:54.973361 10428 tablet_service.cc:1468] Processing CreateTablet for tablet a5935e62f28f40678774f00ee268f351 (DEFAULT_TABLE table=test-workload [id=8debc1751fb84af0a854fd82b5989ba4]), partition=RANGE (key) PARTITION VALUES < 715827882
I20250905 07:47:54.973793 10428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5935e62f28f40678774f00ee268f351. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:54.973971 10584 tablet_service.cc:1468] Processing CreateTablet for tablet 15c647d00a1c45a0966ce65ac0ef0045 (DEFAULT_TABLE table=test-workload [id=8debc1751fb84af0a854fd82b5989ba4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20250905 07:47:54.974427 10584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15c647d00a1c45a0966ce65ac0ef0045. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:54.978610 10669 tablet_bootstrap.cc:492] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423: Bootstrap starting.
I20250905 07:47:54.979497 10670 tablet_bootstrap.cc:492] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d: Bootstrap starting.
I20250905 07:47:54.980728 10669 tablet_bootstrap.cc:654] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:54.981550 10671 tablet_bootstrap.cc:492] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9: Bootstrap starting.
I20250905 07:47:54.981611 10670 tablet_bootstrap.cc:654] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:54.983165 10669 tablet_bootstrap.cc:492] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423: No bootstrap required, opened a new log
I20250905 07:47:54.983323 10669 ts_tablet_manager.cc:1397] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250905 07:47:54.983575 10671 tablet_bootstrap.cc:654] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:54.983727 10670 tablet_bootstrap.cc:492] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d: No bootstrap required, opened a new log
I20250905 07:47:54.983873 10670 ts_tablet_manager.cc:1397] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20250905 07:47:54.984357 10669 raft_consensus.cc:357] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de3982129844172aef574069c768423" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36789 } }
I20250905 07:47:54.984571 10670 raft_consensus.cc:357] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e021f9e358ae427c86dc337066282a1d" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 36149 } }
I20250905 07:47:54.984720 10670 raft_consensus.cc:383] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:54.984827 10670 raft_consensus.cc:738] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e021f9e358ae427c86dc337066282a1d, State: Initialized, Role: FOLLOWER
I20250905 07:47:54.985073 10670 consensus_queue.cc:260] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e021f9e358ae427c86dc337066282a1d" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 36149 } }
I20250905 07:47:54.985173 10669 raft_consensus.cc:383] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:54.985272 10670 raft_consensus.cc:397] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:54.985366 10669 raft_consensus.cc:738] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de3982129844172aef574069c768423, State: Initialized, Role: FOLLOWER
I20250905 07:47:54.985406 10670 raft_consensus.cc:491] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:54.985625 10670 raft_consensus.cc:3058] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:54.985817 10669 consensus_queue.cc:260] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de3982129844172aef574069c768423" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36789 } }
I20250905 07:47:54.986075 10669 raft_consensus.cc:397] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:54.986186 10669 raft_consensus.cc:491] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:54.986349 10669 raft_consensus.cc:3058] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:54.987877 10670 raft_consensus.cc:513] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e021f9e358ae427c86dc337066282a1d" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 36149 } }
I20250905 07:47:54.988188 10670 leader_election.cc:304] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e021f9e358ae427c86dc337066282a1d; no voters:
I20250905 07:47:54.988636 10671 tablet_bootstrap.cc:492] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9: No bootstrap required, opened a new log
I20250905 07:47:54.988705 10670 leader_election.cc:290] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:54.988828 10671 ts_tablet_manager.cc:1397] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.007s
I20250905 07:47:54.988873 10675 raft_consensus.cc:2802] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:54.989336 10669 raft_consensus.cc:513] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de3982129844172aef574069c768423" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36789 } }
I20250905 07:47:54.989617 10669 leader_election.cc:304] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2de3982129844172aef574069c768423; no voters:
I20250905 07:47:54.989701 10671 raft_consensus.cc:357] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42695 } }
I20250905 07:47:54.989902 10671 raft_consensus.cc:383] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:54.990079 10671 raft_consensus.cc:738] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea60c6ba39d4b3a835e8a53732e7bd9, State: Initialized, Role: FOLLOWER
I20250905 07:47:54.990466 10671 consensus_queue.cc:260] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42695 } }
I20250905 07:47:54.990715 10671 raft_consensus.cc:397] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:54.990834 10671 raft_consensus.cc:491] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:54.991014 10671 raft_consensus.cc:3058] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:54.992323 10678 raft_consensus.cc:2802] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:54.992509 10669 leader_election.cc:290] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:54.994354 10671 raft_consensus.cc:513] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42695 } }
I20250905 07:47:54.994657 10671 leader_election.cc:304] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ea60c6ba39d4b3a835e8a53732e7bd9; no voters:
I20250905 07:47:55.000946 10678 raft_consensus.cc:695] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 1 LEADER]: Becoming Leader. State: Replica: 2de3982129844172aef574069c768423, State: Running, Role: LEADER
I20250905 07:47:55.001348 10678 consensus_queue.cc:237] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de3982129844172aef574069c768423" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36789 } }
I20250905 07:47:55.003353 10670 ts_tablet_manager.cc:1428] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d: Time spent starting tablet: real 0.019s user 0.011s sys 0.008s
I20250905 07:47:55.003751 10675 raft_consensus.cc:695] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 1 LEADER]: Becoming Leader. State: Replica: e021f9e358ae427c86dc337066282a1d, State: Running, Role: LEADER
I20250905 07:47:55.004106 10675 consensus_queue.cc:237] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e021f9e358ae427c86dc337066282a1d" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 36149 } }
I20250905 07:47:55.011227 10182 catalog_manager.cc:5582] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2de3982129844172aef574069c768423 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "2de3982129844172aef574069c768423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de3982129844172aef574069c768423" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36789 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:55.020328 10669 ts_tablet_manager.cc:1428] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423: Time spent starting tablet: real 0.037s user 0.011s sys 0.008s
I20250905 07:47:55.037196 10671 leader_election.cc:290] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:55.037329 10681 raft_consensus.cc:2802] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:55.038297 10671 ts_tablet_manager.cc:1428] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9: Time spent starting tablet: real 0.049s user 0.016s sys 0.030s
I20250905 07:47:55.038906 10681 raft_consensus.cc:695] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 1 LEADER]: Becoming Leader. State: Replica: 1ea60c6ba39d4b3a835e8a53732e7bd9, State: Running, Role: LEADER
I20250905 07:47:55.039310 10681 consensus_queue.cc:237] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42695 } }
I20250905 07:47:55.040542 10182 catalog_manager.cc:5582] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d reported cstate change: term changed from 0 to 1, leader changed from <none> to e021f9e358ae427c86dc337066282a1d (127.8.229.65). New cstate: current_term: 1 leader_uuid: "e021f9e358ae427c86dc337066282a1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e021f9e358ae427c86dc337066282a1d" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 36149 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:55.047166 10183 catalog_manager.cc:5582] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ea60c6ba39d4b3a835e8a53732e7bd9 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea60c6ba39d4b3a835e8a53732e7bd9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42695 } health_report { overall_health: HEALTHY } } }
I20250905 07:47:55.069432 9109 master.cc:561] Master@127.8.229.126:40195 shutting down...
I20250905 07:47:55.076836 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:55.077240 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:55.077376 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 386ac99d10cd43849a941982ee5c4e81: stopping tablet replica
I20250905 07:47:55.092599 9109 master.cc:583] Master@127.8.229.126:40195 shutdown complete.
W20250905 07:47:56.036705 10542 heartbeater.cc:646] Failed to heartbeat to 127.8.229.126:40195 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111)
I20250905 07:47:56.553472 10693 raft_consensus.cc:491] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 386ac99d10cd43849a941982ee5c4e81)
I20250905 07:47:56.553712 10693 raft_consensus.cc:513] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:56.553774 10694 raft_consensus.cc:491] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 386ac99d10cd43849a941982ee5c4e81)
I20250905 07:47:56.553972 10694 raft_consensus.cc:513] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:56.554410 10693 leader_election.cc:290] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195), b338d58c7fc64e4c9728743867a30844 (127.8.229.124:34865)
I20250905 07:47:56.554652 10694 leader_election.cc:290] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195), 9379a68a9e76490db703b3c0f2b6caf9 (127.8.229.125:38257)
I20250905 07:47:56.555323 10258 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b338d58c7fc64e4c9728743867a30844" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "9379a68a9e76490db703b3c0f2b6caf9" is_pre_election: true
I20250905 07:47:56.555579 10323 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9379a68a9e76490db703b3c0f2b6caf9" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b338d58c7fc64e4c9728743867a30844" is_pre_election: true
I20250905 07:47:56.555625 10258 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b338d58c7fc64e4c9728743867a30844 in term 1.
I20250905 07:47:56.555797 10323 raft_consensus.cc:2466] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9379a68a9e76490db703b3c0f2b6caf9 in term 1.
I20250905 07:47:56.556177 10298 leader_election.cc:304] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9379a68a9e76490db703b3c0f2b6caf9, b338d58c7fc64e4c9728743867a30844; no voters:
I20250905 07:47:56.556478 10694 raft_consensus.cc:2802] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:47:56.556612 10694 raft_consensus.cc:491] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 386ac99d10cd43849a941982ee5c4e81)
I20250905 07:47:56.556742 10694 raft_consensus.cc:3058] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:47:56.556751 10233 leader_election.cc:304] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9379a68a9e76490db703b3c0f2b6caf9, b338d58c7fc64e4c9728743867a30844; no voters:
I20250905 07:47:56.557101 10693 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20250905 07:47:56.557178 10297 leader_election.cc:336] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195): Network error: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111)
I20250905 07:47:56.557273 10693 raft_consensus.cc:491] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 386ac99d10cd43849a941982ee5c4e81)
I20250905 07:47:56.557387 10693 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 1 FOLLOWER]: Advancing to term 2
W20250905 07:47:56.557677 10233 leader_election.cc:336] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195): Network error: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111)
I20250905 07:47:56.559115 10694 raft_consensus.cc:513] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:56.559533 10693 raft_consensus.cc:513] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:56.559651 10694 leader_election.cc:290] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 2 election: Requested vote from peers 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195), 9379a68a9e76490db703b3c0f2b6caf9 (127.8.229.125:38257)
I20250905 07:47:56.560149 10693 leader_election.cc:290] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [CANDIDATE]: Term 2 election: Requested vote from peers 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195), b338d58c7fc64e4c9728743867a30844 (127.8.229.124:34865)
I20250905 07:47:56.560125 10258 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b338d58c7fc64e4c9728743867a30844" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "9379a68a9e76490db703b3c0f2b6caf9"
I20250905 07:47:56.560492 10258 raft_consensus.cc:2391] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b338d58c7fc64e4c9728743867a30844 in current term 2: Already voted for candidate 9379a68a9e76490db703b3c0f2b6caf9 in this term.
I20250905 07:47:56.560817 10323 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9379a68a9e76490db703b3c0f2b6caf9" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b338d58c7fc64e4c9728743867a30844"
I20250905 07:47:56.561033 10323 raft_consensus.cc:2391] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9379a68a9e76490db703b3c0f2b6caf9 in current term 2: Already voted for candidate b338d58c7fc64e4c9728743867a30844 in this term.
W20250905 07:47:56.561175 10297 leader_election.cc:336] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195): Network error: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111)
I20250905 07:47:56.561327 10297 leader_election.cc:304] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b338d58c7fc64e4c9728743867a30844; no voters: 386ac99d10cd43849a941982ee5c4e81, 9379a68a9e76490db703b3c0f2b6caf9
W20250905 07:47:56.561403 10233 leader_election.cc:336] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195): Network error: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111)
I20250905 07:47:56.561587 10694 raft_consensus.cc:2747] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250905 07:47:56.561640 10233 leader_election.cc:304] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9379a68a9e76490db703b3c0f2b6caf9; no voters: 386ac99d10cd43849a941982ee5c4e81, b338d58c7fc64e4c9728743867a30844
I20250905 07:47:56.561906 10693 raft_consensus.cc:2747] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250905 07:47:58.374500 10705 raft_consensus.cc:491] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:47:58.374691 10705 raft_consensus.cc:513] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:58.375536 10705 leader_election.cc:290] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195), 9379a68a9e76490db703b3c0f2b6caf9 (127.8.229.125:38257)
I20250905 07:47:58.376142 10258 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b338d58c7fc64e4c9728743867a30844" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "9379a68a9e76490db703b3c0f2b6caf9" is_pre_election: true
I20250905 07:47:58.376333 10258 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b338d58c7fc64e4c9728743867a30844 in term 2.
W20250905 07:47:58.376385 10297 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20250905 07:47:58.376793 10298 leader_election.cc:304] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9379a68a9e76490db703b3c0f2b6caf9, b338d58c7fc64e4c9728743867a30844; no voters:
I20250905 07:47:58.377079 10705 raft_consensus.cc:2802] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250905 07:47:58.377218 10705 raft_consensus.cc:491] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:47:58.377341 10705 raft_consensus.cc:3058] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 2 FOLLOWER]: Advancing to term 3
W20250905 07:47:58.377738 10297 leader_election.cc:336] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195): Network error: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111)
I20250905 07:47:58.379267 10705 raft_consensus.cc:513] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:58.379849 10705 leader_election.cc:290] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 3 election: Requested vote from peers 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195), 9379a68a9e76490db703b3c0f2b6caf9 (127.8.229.125:38257)
I20250905 07:47:58.380501 10258 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b338d58c7fc64e4c9728743867a30844" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "9379a68a9e76490db703b3c0f2b6caf9"
I20250905 07:47:58.380692 10258 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 2 FOLLOWER]: Advancing to term 3
W20250905 07:47:58.381059 10297 leader_election.cc:336] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 386ac99d10cd43849a941982ee5c4e81 (127.8.229.126:40195): Network error: Client connection negotiation failed: client connection to 127.8.229.126:40195: connect: Connection refused (error 111)
I20250905 07:47:58.382596 10258 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b338d58c7fc64e4c9728743867a30844 in term 3.
I20250905 07:47:58.382956 10298 leader_election.cc:304] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9379a68a9e76490db703b3c0f2b6caf9, b338d58c7fc64e4c9728743867a30844; no voters: 386ac99d10cd43849a941982ee5c4e81
I20250905 07:47:58.383194 10705 raft_consensus.cc:2802] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:47:58.383579 10705 raft_consensus.cc:695] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 3 LEADER]: Becoming Leader. State: Replica: b338d58c7fc64e4c9728743867a30844, State: Running, Role: LEADER
I20250905 07:47:58.383843 10705 consensus_queue.cc:237] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } }
I20250905 07:47:58.384976 10708 sys_catalog.cc:455] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b338d58c7fc64e4c9728743867a30844. Latest consensus state: current_term: 3 leader_uuid: "b338d58c7fc64e4c9728743867a30844" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "386ac99d10cd43849a941982ee5c4e81" member_type: VOTER last_known_addr { host: "127.8.229.126" port: 40195 } } peers { permanent_uuid: "9379a68a9e76490db703b3c0f2b6caf9" member_type: VOTER last_known_addr { host: "127.8.229.125" port: 38257 } } peers { permanent_uuid: "b338d58c7fc64e4c9728743867a30844" member_type: VOTER last_known_addr { host: "127.8.229.124" port: 34865 } } }
I20250905 07:47:58.385203 10708 sys_catalog.cc:458] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:58.385690 10710 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:47:58.387383 10710 catalog_manager.cc:671] Loaded metadata for table test-workload [id=8debc1751fb84af0a854fd82b5989ba4]
I20250905 07:47:58.388757 10710 tablet_loader.cc:96] loaded metadata for tablet 15c647d00a1c45a0966ce65ac0ef0045 (table test-workload [id=8debc1751fb84af0a854fd82b5989ba4])
I20250905 07:47:58.389089 10710 tablet_loader.cc:96] loaded metadata for tablet 49d138ad00434a6ea2a10942ff2dec27 (table test-workload [id=8debc1751fb84af0a854fd82b5989ba4])
I20250905 07:47:58.389398 10710 tablet_loader.cc:96] loaded metadata for tablet a5935e62f28f40678774f00ee268f351 (table test-workload [id=8debc1751fb84af0a854fd82b5989ba4])
I20250905 07:47:58.389621 10710 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:47:58.390417 10710 catalog_manager.cc:1261] Loaded cluster ID: ba4ef1442d484da78f1062b53cb82a9c
I20250905 07:47:58.390491 10710 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:47:58.391604 10710 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:47:58.392369 10710 catalog_manager.cc:5966] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: Loaded TSK: 0
I20250905 07:47:58.393006 10710 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:47:58.532658 10378 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:47:58.532789 10378 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:47:58.543156 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:47:58.553560 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:58.553798 9109 tablet_replica.cc:331] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d: stopping tablet replica
I20250905 07:47:58.553987 9109 raft_consensus.cc:2241] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:58.554178 9109 raft_consensus.cc:2270] T a5935e62f28f40678774f00ee268f351 P e021f9e358ae427c86dc337066282a1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:58.567596 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:47:58.570295 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:47:58.579107 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:58.579358 9109 tablet_replica.cc:331] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423: stopping tablet replica
I20250905 07:47:58.579594 9109 raft_consensus.cc:2241] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:58.579777 9109 raft_consensus.cc:2270] T 49d138ad00434a6ea2a10942ff2dec27 P 2de3982129844172aef574069c768423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:58.593292 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:47:58.595958 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:47:58.604094 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:47:58.604357 9109 tablet_replica.cc:331] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9: stopping tablet replica
I20250905 07:47:58.604542 9109 raft_consensus.cc:2241] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:47:58.604718 9109 raft_consensus.cc:2270] T 15c647d00a1c45a0966ce65ac0ef0045 P 1ea60c6ba39d4b3a835e8a53732e7bd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:58.618050 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:47:58.620885 9109 master.cc:561] Master@127.8.229.125:38257 shutting down...
I20250905 07:47:58.629093 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:47:58.629333 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:58.629482 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 9379a68a9e76490db703b3c0f2b6caf9: stopping tablet replica
I20250905 07:47:58.643216 9109 master.cc:583] Master@127.8.229.125:38257 shutdown complete.
I20250905 07:47:58.646910 9109 master.cc:561] Master@127.8.229.124:34865 shutting down...
I20250905 07:47:58.654924 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 3 LEADER]: Raft consensus shutting down.
I20250905 07:47:58.655258 9109 pending_rounds.cc:70] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: Trying to abort 1 pending ops.
I20250905 07:47:58.655467 9109 pending_rounds.cc:77] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: Aborting op as it isn't in flight: id { term: 3 index: 11 } timestamp: 7196911527462100992 op_type: NO_OP noop_request { }
I20250905 07:47:58.655637 9109 raft_consensus.cc:2887] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:47:58.655751 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:47:58.655947 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P b338d58c7fc64e4c9728743867a30844: stopping tablet replica
I20250905 07:47:58.669823 9109 master.cc:583] Master@127.8.229.124:34865 shutdown complete.
[ OK ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing (6321 ms)
[ RUN ] AutoRebalancerTest.MovesScheduledIfAddTserver
I20250905 07:47:58.682058 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:37117
I20250905 07:47:58.682597 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:58.685003 10711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.685231 10712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.685776 10714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:58.686269 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:58.686545 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:58.686630 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:58.686688 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058478686669 us; error 0 us; skew 500 ppm
I20250905 07:47:58.686857 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:58.688108 9109 webserver.cc:480] Webserver started at http://127.8.229.126:42171/ using document root <none> and password file <none>
I20250905 07:47:58.688344 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:58.688437 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:58.688601 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:58.689271 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "186be7a143ec484e8b15d258a291d2da"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.691777 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:47:58.693459 10719 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.693819 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:47:58.693948 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "186be7a143ec484e8b15d258a291d2da"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.694069 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:58.712806 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:58.713428 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:58.727061 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:37117
I20250905 07:47:58.727097 10770 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:37117 every 8 connection(s)
I20250905 07:47:58.728437 10771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:58.732250 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da: Bootstrap starting.
I20250905 07:47:58.733584 10771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da: Neither blocks nor log segments found. Creating new log.
I20250905 07:47:58.735093 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da: No bootstrap required, opened a new log
I20250905 07:47:58.735770 10771 raft_consensus.cc:357] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186be7a143ec484e8b15d258a291d2da" member_type: VOTER }
I20250905 07:47:58.735903 10771 raft_consensus.cc:383] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:47:58.735998 10771 raft_consensus.cc:738] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186be7a143ec484e8b15d258a291d2da, State: Initialized, Role: FOLLOWER
I20250905 07:47:58.736212 10771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186be7a143ec484e8b15d258a291d2da" member_type: VOTER }
I20250905 07:47:58.736349 10771 raft_consensus.cc:397] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:47:58.736429 10771 raft_consensus.cc:491] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:47:58.736541 10771 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:47:58.738137 10771 raft_consensus.cc:513] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186be7a143ec484e8b15d258a291d2da" member_type: VOTER }
I20250905 07:47:58.738353 10771 leader_election.cc:304] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186be7a143ec484e8b15d258a291d2da; no voters:
I20250905 07:47:58.738729 10771 leader_election.cc:290] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:47:58.738831 10774 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:47:58.739356 10774 raft_consensus.cc:695] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 1 LEADER]: Becoming Leader. State: Replica: 186be7a143ec484e8b15d258a291d2da, State: Running, Role: LEADER
I20250905 07:47:58.739619 10774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186be7a143ec484e8b15d258a291d2da" member_type: VOTER }
I20250905 07:47:58.739668 10771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:47:58.740543 10775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "186be7a143ec484e8b15d258a291d2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186be7a143ec484e8b15d258a291d2da" member_type: VOTER } }
I20250905 07:47:58.740579 10776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186be7a143ec484e8b15d258a291d2da. Latest consensus state: current_term: 1 leader_uuid: "186be7a143ec484e8b15d258a291d2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186be7a143ec484e8b15d258a291d2da" member_type: VOTER } }
I20250905 07:47:58.740751 10775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:58.740775 10776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [sys.catalog]: This master's current role is: LEADER
I20250905 07:47:58.741894 10779 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:47:58.743379 10779 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:47:58.745165 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:47:58.746227 10779 catalog_manager.cc:1349] Generated new cluster ID: 6a984c49673f4bcdadf9291582040de9
I20250905 07:47:58.746309 10779 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:47:58.755352 10779 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:47:58.756047 10779 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:47:58.769284 10779 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da: Generated new TSK 0
I20250905 07:47:58.769510 10779 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:47:58.777779 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:58.780536 10792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.780764 10793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.781760 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:58.781916 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:58.782192 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:58.782255 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:58.782312 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058478782287 us; error 0 us; skew 500 ppm
I20250905 07:47:58.782444 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:58.783550 9109 webserver.cc:480] Webserver started at http://127.8.229.65:36323/ using document root <none> and password file <none>
I20250905 07:47:58.783748 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:58.783828 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:58.783936 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:58.784585 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "745f84b573b4476190673e98e17c9cb1"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.786921 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:47:58.788568 10800 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.788916 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:47:58.789008 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "745f84b573b4476190673e98e17c9cb1"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.789161 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:58.802887 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:58.803514 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:58.804090 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:58.804949 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:58.805011 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.805119 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:58.805182 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.818661 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:35271
I20250905 07:47:58.818681 10862 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:35271 every 8 connection(s)
I20250905 07:47:58.820362 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:58.823259 10868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.823812 10867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.824421 10870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:58.824928 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:58.825197 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:58.825297 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:58.825361 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058478825336 us; error 0 us; skew 500 ppm
I20250905 07:47:58.825521 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:58.826678 9109 webserver.cc:480] Webserver started at http://127.8.229.66:35009/ using document root <none> and password file <none>
I20250905 07:47:58.826893 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:58.826974 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:58.827134 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:58.827788 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "4dbb1b1653474259b304dcf018e6a33e"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.830127 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:47:58.831713 10875 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.832019 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:47:58.832151 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "4dbb1b1653474259b304dcf018e6a33e"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.832259 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:58.834717 10863 heartbeater.cc:344] Connected to a master server at 127.8.229.126:37117
I20250905 07:47:58.834846 10863 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:58.835250 10863 heartbeater.cc:507] Master 127.8.229.126:37117 requested a full tablet report, sending...
I20250905 07:47:58.836161 10736 ts_manager.cc:194] Registered new tserver with Master: 745f84b573b4476190673e98e17c9cb1 (127.8.229.65:35271)
I20250905 07:47:58.837217 10736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54414
I20250905 07:47:58.849089 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:58.849655 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:58.850201 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:58.851006 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:58.851095 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.851228 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:58.851303 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.864101 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:35679
I20250905 07:47:58.864144 10937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:35679 every 8 connection(s)
I20250905 07:47:58.865692 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:47:58.868687 10941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.868760 10942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:47:58.871762 10944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:47:58.871963 10938 heartbeater.cc:344] Connected to a master server at 127.8.229.126:37117
I20250905 07:47:58.872022 9109 server_base.cc:1047] running on GCE node
I20250905 07:47:58.872134 10938 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:58.872339 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:47:58.872426 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:47:58.872512 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058478872468 us; error 0 us; skew 500 ppm
I20250905 07:47:58.872582 10938 heartbeater.cc:507] Master 127.8.229.126:37117 requested a full tablet report, sending...
I20250905 07:47:58.872685 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:47:58.873519 10736 ts_manager.cc:194] Registered new tserver with Master: 4dbb1b1653474259b304dcf018e6a33e (127.8.229.66:35679)
I20250905 07:47:58.873912 9109 webserver.cc:480] Webserver started at http://127.8.229.67:38479/ using document root <none> and password file <none>
I20250905 07:47:58.874118 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:47:58.874224 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:47:58.874363 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:47:58.874366 10736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54422
I20250905 07:47:58.875118 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "0eebf4a15ea54cd1886d93620717ea86"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.877470 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:47:58.879029 10949 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.879421 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:47:58.879567 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "0eebf4a15ea54cd1886d93620717ea86"
format_stamp: "Formatted at 2025-09-05 07:47:58 on dist-test-slave-pgzt"
I20250905 07:47:58.879702 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:47:58.891513 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:47:58.892099 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:47:58.892627 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:47:58.893404 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:47:58.893481 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.901880 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:47:58.901952 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:47:58.912470 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:43763
I20250905 07:47:58.912492 11011 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:43763 every 8 connection(s)
I20250905 07:47:58.918565 11012 heartbeater.cc:344] Connected to a master server at 127.8.229.126:37117
I20250905 07:47:58.918702 11012 heartbeater.cc:461] Registering TS with master...
I20250905 07:47:58.919013 11012 heartbeater.cc:507] Master 127.8.229.126:37117 requested a full tablet report, sending...
I20250905 07:47:58.919800 10736 ts_manager.cc:194] Registered new tserver with Master: 0eebf4a15ea54cd1886d93620717ea86 (127.8.229.67:43763)
I20250905 07:47:58.920037 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00676358s
I20250905 07:47:58.920516 10736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54438
I20250905 07:47:59.838371 10863 heartbeater.cc:499] Master 127.8.229.126:37117 was elected leader, sending a full tablet report...
I20250905 07:47:59.875639 10938 heartbeater.cc:499] Master 127.8.229.126:37117 was elected leader, sending a full tablet report...
I20250905 07:47:59.921675 11012 heartbeater.cc:499] Master 127.8.229.126:37117 was elected leader, sending a full tablet report...
I20250905 07:47:59.947526 9109 test_util.cc:276] Using random seed: 244055777
I20250905 07:47:59.961470 10736 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54444:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:47:59.962142 10736 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:47:59.995784 10976 tablet_service.cc:1468] Processing CreateTablet for tablet 816721c5807d45968757f82150ee457b (DEFAULT_TABLE table=test-workload [id=dab7422ec1eb4cdca85e5112969dcca3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20250905 07:47:59.996341 10976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 816721c5807d45968757f82150ee457b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:59.996693 10828 tablet_service.cc:1468] Processing CreateTablet for tablet fac73c227d6746f2b01f416bfa440f5c (DEFAULT_TABLE table=test-workload [id=dab7422ec1eb4cdca85e5112969dcca3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20250905 07:47:59.997020 10828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac73c227d6746f2b01f416bfa440f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:59.995347 10977 tablet_service.cc:1468] Processing CreateTablet for tablet fac73c227d6746f2b01f416bfa440f5c (DEFAULT_TABLE table=test-workload [id=dab7422ec1eb4cdca85e5112969dcca3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20250905 07:47:59.998064 10827 tablet_service.cc:1468] Processing CreateTablet for tablet 816721c5807d45968757f82150ee457b (DEFAULT_TABLE table=test-workload [id=dab7422ec1eb4cdca85e5112969dcca3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20250905 07:47:59.998196 10977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac73c227d6746f2b01f416bfa440f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:47:59.998441 10827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 816721c5807d45968757f82150ee457b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:00.004464 10903 tablet_service.cc:1468] Processing CreateTablet for tablet fac73c227d6746f2b01f416bfa440f5c (DEFAULT_TABLE table=test-workload [id=dab7422ec1eb4cdca85e5112969dcca3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20250905 07:48:00.004693 10902 tablet_service.cc:1468] Processing CreateTablet for tablet 816721c5807d45968757f82150ee457b (DEFAULT_TABLE table=test-workload [id=dab7422ec1eb4cdca85e5112969dcca3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20250905 07:48:00.004902 10903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac73c227d6746f2b01f416bfa440f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:00.005187 10902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 816721c5807d45968757f82150ee457b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:00.009300 11032 tablet_bootstrap.cc:492] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86: Bootstrap starting.
I20250905 07:48:00.011871 11032 tablet_bootstrap.cc:654] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:00.012136 11033 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: Bootstrap starting.
I20250905 07:48:00.014402 11034 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: Bootstrap starting.
I20250905 07:48:00.014442 11033 tablet_bootstrap.cc:654] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:00.015256 11032 tablet_bootstrap.cc:492] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86: No bootstrap required, opened a new log
I20250905 07:48:00.015396 11032 ts_tablet_manager.cc:1397] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250905 07:48:00.015993 11032 raft_consensus.cc:357] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.016165 11032 raft_consensus.cc:383] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:00.016230 11032 raft_consensus.cc:738] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eebf4a15ea54cd1886d93620717ea86, State: Initialized, Role: FOLLOWER
I20250905 07:48:00.016422 11032 consensus_queue.cc:260] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.016607 11034 tablet_bootstrap.cc:654] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:00.018343 11032 ts_tablet_manager.cc:1428] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250905 07:48:00.018610 11032 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86: Bootstrap starting.
I20250905 07:48:00.020364 11032 tablet_bootstrap.cc:654] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:00.025681 11033 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: No bootstrap required, opened a new log
I20250905 07:48:00.025681 11034 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: No bootstrap required, opened a new log
I20250905 07:48:00.025875 11034 ts_tablet_manager.cc:1397] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20250905 07:48:00.025875 11033 ts_tablet_manager.cc:1397] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20250905 07:48:00.026666 11033 raft_consensus.cc:357] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.026666 11034 raft_consensus.cc:357] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.026880 11033 raft_consensus.cc:383] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:00.026890 11034 raft_consensus.cc:383] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:00.026952 11033 raft_consensus.cc:738] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 745f84b573b4476190673e98e17c9cb1, State: Initialized, Role: FOLLOWER
I20250905 07:48:00.026955 11034 raft_consensus.cc:738] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbb1b1653474259b304dcf018e6a33e, State: Initialized, Role: FOLLOWER
I20250905 07:48:00.027230 11033 consensus_queue.cc:260] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.027230 11034 consensus_queue.cc:260] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.028337 11034 ts_tablet_manager.cc:1428] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:48:00.029011 11034 tablet_bootstrap.cc:492] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e: Bootstrap starting.
I20250905 07:48:00.029428 11032 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86: No bootstrap required, opened a new log
I20250905 07:48:00.029564 11032 ts_tablet_manager.cc:1397] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20250905 07:48:00.030516 11032 raft_consensus.cc:357] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.030701 11032 raft_consensus.cc:383] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:00.030757 11032 raft_consensus.cc:738] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eebf4a15ea54cd1886d93620717ea86, State: Initialized, Role: FOLLOWER
I20250905 07:48:00.030974 11032 consensus_queue.cc:260] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.033111 11034 tablet_bootstrap.cc:654] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:00.033579 11032 ts_tablet_manager.cc:1428] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250905 07:48:00.034780 11033 ts_tablet_manager.cc:1428] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: Time spent starting tablet: real 0.009s user 0.000s sys 0.008s
I20250905 07:48:00.035033 11033 tablet_bootstrap.cc:492] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1: Bootstrap starting.
I20250905 07:48:00.035713 11034 tablet_bootstrap.cc:492] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e: No bootstrap required, opened a new log
I20250905 07:48:00.035868 11034 ts_tablet_manager.cc:1397] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250905 07:48:00.036597 11034 raft_consensus.cc:357] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.036732 11033 tablet_bootstrap.cc:654] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:00.036795 11034 raft_consensus.cc:383] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:00.036854 11034 raft_consensus.cc:738] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbb1b1653474259b304dcf018e6a33e, State: Initialized, Role: FOLLOWER
I20250905 07:48:00.037093 11034 consensus_queue.cc:260] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.037569 11034 ts_tablet_manager.cc:1428] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:48:00.044524 11033 tablet_bootstrap.cc:492] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1: No bootstrap required, opened a new log
I20250905 07:48:00.044661 11033 ts_tablet_manager.cc:1397] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20250905 07:48:00.045221 11033 raft_consensus.cc:357] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.045357 11033 raft_consensus.cc:383] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:00.045397 11033 raft_consensus.cc:738] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 745f84b573b4476190673e98e17c9cb1, State: Initialized, Role: FOLLOWER
I20250905 07:48:00.045564 11033 consensus_queue.cc:260] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.045945 11033 ts_tablet_manager.cc:1428] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:48:00.053658 11041 raft_consensus.cc:491] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:48:00.053812 11041 raft_consensus.cc:513] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.054513 11041 leader_election.cc:290] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dbb1b1653474259b304dcf018e6a33e (127.8.229.66:35679), 0eebf4a15ea54cd1886d93620717ea86 (127.8.229.67:43763)
I20250905 07:48:00.060413 10913 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "816721c5807d45968757f82150ee457b" candidate_uuid: "745f84b573b4476190673e98e17c9cb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb1b1653474259b304dcf018e6a33e" is_pre_election: true
I20250905 07:48:00.060582 10987 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "816721c5807d45968757f82150ee457b" candidate_uuid: "745f84b573b4476190673e98e17c9cb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eebf4a15ea54cd1886d93620717ea86" is_pre_election: true
I20250905 07:48:00.060657 10913 raft_consensus.cc:2466] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 745f84b573b4476190673e98e17c9cb1 in term 0.
I20250905 07:48:00.060811 10987 raft_consensus.cc:2466] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 745f84b573b4476190673e98e17c9cb1 in term 0.
I20250905 07:48:00.061096 10802 leader_election.cc:304] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbb1b1653474259b304dcf018e6a33e, 745f84b573b4476190673e98e17c9cb1; no voters:
I20250905 07:48:00.061367 11041 raft_consensus.cc:2802] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:48:00.061442 11041 raft_consensus.cc:491] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:48:00.061494 11041 raft_consensus.cc:3058] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:00.063524 11041 raft_consensus.cc:513] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.064029 11041 leader_election.cc:290] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dbb1b1653474259b304dcf018e6a33e (127.8.229.66:35679), 0eebf4a15ea54cd1886d93620717ea86 (127.8.229.67:43763)
I20250905 07:48:00.064412 10987 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "816721c5807d45968757f82150ee457b" candidate_uuid: "745f84b573b4476190673e98e17c9cb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eebf4a15ea54cd1886d93620717ea86"
I20250905 07:48:00.064540 10987 raft_consensus.cc:3058] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:00.066324 10987 raft_consensus.cc:2466] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 745f84b573b4476190673e98e17c9cb1 in term 1.
I20250905 07:48:00.066556 10913 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "816721c5807d45968757f82150ee457b" candidate_uuid: "745f84b573b4476190673e98e17c9cb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb1b1653474259b304dcf018e6a33e"
I20250905 07:48:00.066718 10913 raft_consensus.cc:3058] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:00.066684 10803 leader_election.cc:304] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eebf4a15ea54cd1886d93620717ea86, 745f84b573b4476190673e98e17c9cb1; no voters:
I20250905 07:48:00.066963 11041 raft_consensus.cc:2802] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:48:00.067386 11041 raft_consensus.cc:695] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 1 LEADER]: Becoming Leader. State: Replica: 745f84b573b4476190673e98e17c9cb1, State: Running, Role: LEADER
I20250905 07:48:00.067664 11041 consensus_queue.cc:237] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } }
I20250905 07:48:00.069088 10913 raft_consensus.cc:2466] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 745f84b573b4476190673e98e17c9cb1 in term 1.
I20250905 07:48:00.070258 10735 catalog_manager.cc:5582] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 745f84b573b4476190673e98e17c9cb1 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "745f84b573b4476190673e98e17c9cb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20250905 07:48:00.083602 11040 raft_consensus.cc:491] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:48:00.083796 11040 raft_consensus.cc:513] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.084600 11040 leader_election.cc:290] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 745f84b573b4476190673e98e17c9cb1 (127.8.229.65:35271), 0eebf4a15ea54cd1886d93620717ea86 (127.8.229.67:43763)
I20250905 07:48:00.091912 10987 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "fac73c227d6746f2b01f416bfa440f5c" candidate_uuid: "4dbb1b1653474259b304dcf018e6a33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eebf4a15ea54cd1886d93620717ea86" is_pre_election: true
I20250905 07:48:00.091912 10838 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "fac73c227d6746f2b01f416bfa440f5c" candidate_uuid: "4dbb1b1653474259b304dcf018e6a33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745f84b573b4476190673e98e17c9cb1" is_pre_election: true
I20250905 07:48:00.092180 10987 raft_consensus.cc:2466] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbb1b1653474259b304dcf018e6a33e in term 0.
I20250905 07:48:00.092188 10838 raft_consensus.cc:2466] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbb1b1653474259b304dcf018e6a33e in term 0.
I20250905 07:48:00.092547 10878 leader_election.cc:304] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eebf4a15ea54cd1886d93620717ea86, 4dbb1b1653474259b304dcf018e6a33e; no voters:
I20250905 07:48:00.092790 11040 raft_consensus.cc:2802] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:48:00.092870 11040 raft_consensus.cc:491] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:48:00.092937 11040 raft_consensus.cc:3058] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:00.094661 11040 raft_consensus.cc:513] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.095142 11040 leader_election.cc:290] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [CANDIDATE]: Term 1 election: Requested vote from peers 745f84b573b4476190673e98e17c9cb1 (127.8.229.65:35271), 0eebf4a15ea54cd1886d93620717ea86 (127.8.229.67:43763)
I20250905 07:48:00.095605 10838 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "fac73c227d6746f2b01f416bfa440f5c" candidate_uuid: "4dbb1b1653474259b304dcf018e6a33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745f84b573b4476190673e98e17c9cb1"
I20250905 07:48:00.095588 10987 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "fac73c227d6746f2b01f416bfa440f5c" candidate_uuid: "4dbb1b1653474259b304dcf018e6a33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eebf4a15ea54cd1886d93620717ea86"
I20250905 07:48:00.095767 10838 raft_consensus.cc:3058] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:00.095793 10987 raft_consensus.cc:3058] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:00.097553 10987 raft_consensus.cc:2466] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbb1b1653474259b304dcf018e6a33e in term 1.
I20250905 07:48:00.097553 10838 raft_consensus.cc:2466] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbb1b1653474259b304dcf018e6a33e in term 1.
I20250905 07:48:00.098075 10878 leader_election.cc:304] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0eebf4a15ea54cd1886d93620717ea86, 4dbb1b1653474259b304dcf018e6a33e, 745f84b573b4476190673e98e17c9cb1; no voters:
I20250905 07:48:00.098428 11040 raft_consensus.cc:2802] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:48:00.098850 11040 raft_consensus.cc:695] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 LEADER]: Becoming Leader. State: Replica: 4dbb1b1653474259b304dcf018e6a33e, State: Running, Role: LEADER
I20250905 07:48:00.099118 11040 consensus_queue.cc:237] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } }
I20250905 07:48:00.101509 10735 catalog_manager.cc:5582] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dbb1b1653474259b304dcf018e6a33e (127.8.229.66). New cstate: current_term: 1 leader_uuid: "4dbb1b1653474259b304dcf018e6a33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } health_report { overall_health: HEALTHY } } }
I20250905 07:48:00.502417 11041 consensus_queue.cc:1035] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:48:00.508803 11041 consensus_queue.cc:1035] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:48:00.580549 11040 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:48:00.584619 11059 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:48:00.745096 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:00.745620 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:00.745709 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:02.746081 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:02.746636 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:02.746752 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:04.747218 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:04.747850 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:04.747946 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:06.748395 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:06.748929 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:06.749022 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:08.749433 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:08.750003 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:08.750120 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:10.750489 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:10.751024 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:10.751134 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:11.178753 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:11.181519 11068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:11.181789 11069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:11.182271 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:11.185104 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:11.186036 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:11.186101 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:11.186127 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058491186124 us; error 0 us; skew 500 ppm
I20250905 07:48:11.186252 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:11.187563 9109 webserver.cc:480] Webserver started at http://127.8.229.68:37927/ using document root <none> and password file <none>
I20250905 07:48:11.187789 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:11.187867 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:11.187981 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:11.188635 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "5040468934274f999921dbbe91769dec"
format_stamp: "Formatted at 2025-09-05 07:48:11 on dist-test-slave-pgzt"
I20250905 07:48:11.191146 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:48:11.192894 11076 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:11.193306 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:48:11.193434 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "5040468934274f999921dbbe91769dec"
format_stamp: "Formatted at 2025-09-05 07:48:11 on dist-test-slave-pgzt"
I20250905 07:48:11.193540 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:11.202540 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:11.203050 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:11.203613 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:11.204416 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:11.204495 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:11.204564 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:11.204620 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:11.218945 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:36931
I20250905 07:48:11.218982 11138 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:36931 every 8 connection(s)
I20250905 07:48:11.225908 11139 heartbeater.cc:344] Connected to a master server at 127.8.229.126:37117
I20250905 07:48:11.226064 11139 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:11.226469 11139 heartbeater.cc:507] Master 127.8.229.126:37117 requested a full tablet report, sending...
I20250905 07:48:11.227360 10735 ts_manager.cc:194] Registered new tserver with Master: 5040468934274f999921dbbe91769dec (127.8.229.68:36931)
I20250905 07:48:11.228251 10735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60186
I20250905 07:48:11.766547 10913 consensus_queue.cc:237] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: true } }
I20250905 07:48:11.769444 10838 raft_consensus.cc:1273] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Refusing update from remote peer 4dbb1b1653474259b304dcf018e6a33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:48:11.770063 11143 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:48:11.770538 10987 raft_consensus.cc:1273] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Refusing update from remote peer 4dbb1b1653474259b304dcf018e6a33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:48:11.771111 11143 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:48:11.774045 11148 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5040468934274f999921dbbe91769dec (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: true } } }
I20250905 07:48:11.774932 10987 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5040468934274f999921dbbe91769dec (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: true } } }
I20250905 07:48:11.778475 10838 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5040468934274f999921dbbe91769dec (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: true } } }
I20250905 07:48:11.779425 10735 catalog_manager.cc:5582] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e reported cstate change: config changed from index -1 to 2, NON_VOTER 5040468934274f999921dbbe91769dec (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "4dbb1b1653474259b304dcf018e6a33e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250905 07:48:11.780769 10878 consensus_peers.cc:489] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e -> Peer 5040468934274f999921dbbe91769dec (127.8.229.68:36931): Couldn't send request to peer 5040468934274f999921dbbe91769dec. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fac73c227d6746f2b01f416bfa440f5c. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:48:12.229508 11139 heartbeater.cc:499] Master 127.8.229.126:37117 was elected leader, sending a full tablet report...
I20250905 07:48:12.259289 11158 ts_tablet_manager.cc:927] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: Initiating tablet copy from peer 4dbb1b1653474259b304dcf018e6a33e (127.8.229.66:35679)
I20250905 07:48:12.260445 11158 tablet_copy_client.cc:323] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:35679
I20250905 07:48:12.267994 10923 tablet_copy_service.cc:140] P 4dbb1b1653474259b304dcf018e6a33e: Received BeginTabletCopySession request for tablet fac73c227d6746f2b01f416bfa440f5c from peer 5040468934274f999921dbbe91769dec ({username='slave'} at 127.0.0.1:39546)
I20250905 07:48:12.268270 10923 tablet_copy_service.cc:161] P 4dbb1b1653474259b304dcf018e6a33e: Beginning new tablet copy session on tablet fac73c227d6746f2b01f416bfa440f5c from peer 5040468934274f999921dbbe91769dec at {username='slave'} at 127.0.0.1:39546: session id = 5040468934274f999921dbbe91769dec-fac73c227d6746f2b01f416bfa440f5c
I20250905 07:48:12.271737 10923 tablet_copy_source_session.cc:215] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:48:12.273609 11158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac73c227d6746f2b01f416bfa440f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:12.279624 11158 tablet_copy_client.cc:806] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: tablet copy: Starting download of 0 data blocks...
I20250905 07:48:12.279999 11158 tablet_copy_client.cc:670] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: tablet copy: Starting download of 1 WAL segments...
I20250905 07:48:12.282389 11158 tablet_copy_client.cc:538] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:48:12.286393 11158 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: Bootstrap starting.
I20250905 07:48:12.294533 11158 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:48:12.295053 11158 tablet_bootstrap.cc:492] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: Bootstrap complete.
I20250905 07:48:12.295325 11158 ts_tablet_manager.cc:1397] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20250905 07:48:12.296172 11158 raft_consensus.cc:357] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: true } }
I20250905 07:48:12.296375 11158 raft_consensus.cc:738] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5040468934274f999921dbbe91769dec, State: Initialized, Role: LEARNER
I20250905 07:48:12.296607 11158 consensus_queue.cc:260] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: true } }
I20250905 07:48:12.297365 11158 ts_tablet_manager.cc:1428] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20250905 07:48:12.298156 10923 tablet_copy_service.cc:342] P 4dbb1b1653474259b304dcf018e6a33e: Request end of tablet copy session 5040468934274f999921dbbe91769dec-fac73c227d6746f2b01f416bfa440f5c received from {username='slave'} at 127.0.0.1:39546
I20250905 07:48:12.298362 10923 tablet_copy_service.cc:434] P 4dbb1b1653474259b304dcf018e6a33e: ending tablet copy session 5040468934274f999921dbbe91769dec-fac73c227d6746f2b01f416bfa440f5c on tablet fac73c227d6746f2b01f416bfa440f5c with peer 5040468934274f999921dbbe91769dec
I20250905 07:48:12.662559 11114 raft_consensus.cc:1215] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250905 07:48:12.751511 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:12.752241 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:12.752390 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:13.170080 11165 raft_consensus.cc:1062] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: attempting to promote NON_VOTER 5040468934274f999921dbbe91769dec to VOTER
I20250905 07:48:13.170892 11165 consensus_queue.cc:237] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } }
I20250905 07:48:13.173035 11114 raft_consensus.cc:1273] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 LEARNER]: Refusing update from remote peer 4dbb1b1653474259b304dcf018e6a33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:48:13.173532 10987 raft_consensus.cc:1273] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Refusing update from remote peer 4dbb1b1653474259b304dcf018e6a33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:48:13.173593 11167 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:48:13.174047 10838 raft_consensus.cc:1273] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Refusing update from remote peer 4dbb1b1653474259b304dcf018e6a33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:48:13.174113 11164 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:48:13.174595 11167 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:48:13.177954 11167 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5040468934274f999921dbbe91769dec (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.178452 10838 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5040468934274f999921dbbe91769dec (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.178615 11114 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5040468934274f999921dbbe91769dec (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.182952 10734 catalog_manager.cc:5582] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 reported cstate change: config changed from index 2 to 3, 5040468934274f999921dbbe91769dec (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dbb1b1653474259b304dcf018e6a33e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.184001 10987 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5040468934274f999921dbbe91769dec (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "745f84b573b4476190673e98e17c9cb1" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35271 } attrs { replace: true } } peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.194064 10913 consensus_queue.cc:237] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } }
I20250905 07:48:13.195921 10987 raft_consensus.cc:1273] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Refusing update from remote peer 4dbb1b1653474259b304dcf018e6a33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:48:13.195935 11114 raft_consensus.cc:1273] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 FOLLOWER]: Refusing update from remote peer 4dbb1b1653474259b304dcf018e6a33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:48:13.196535 11167 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:48:13.196743 11165 consensus_queue.cc:1035] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:48:13.198992 11164 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 745f84b573b4476190673e98e17c9cb1 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.200012 10987 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 745f84b573b4476190673e98e17c9cb1 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.200062 11114 raft_consensus.cc:2953] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 745f84b573b4476190673e98e17c9cb1 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } } }
I20250905 07:48:13.202931 10721 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet fac73c227d6746f2b01f416bfa440f5c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:48:13.204109 10733 catalog_manager.cc:5582] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e reported cstate change: config changed from index 3 to 4, VOTER 745f84b573b4476190673e98e17c9cb1 (127.8.229.65) evicted. New cstate: current_term: 1 leader_uuid: "4dbb1b1653474259b304dcf018e6a33e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0eebf4a15ea54cd1886d93620717ea86" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 43763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dbb1b1653474259b304dcf018e6a33e" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 35679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5040468934274f999921dbbe91769dec" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36931 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:48:13.208070 10828 tablet_service.cc:1515] Processing DeleteTablet for tablet fac73c227d6746f2b01f416bfa440f5c with delete_type TABLET_DATA_TOMBSTONED (TS 745f84b573b4476190673e98e17c9cb1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35914
I20250905 07:48:13.208865 11177 tablet_replica.cc:331] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: stopping tablet replica
I20250905 07:48:13.209036 11177 raft_consensus.cc:2241] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:48:13.209285 11177 raft_consensus.cc:2270] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:13.210006 11177 ts_tablet_manager.cc:1905] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:48:13.214237 11177 ts_tablet_manager.cc:1918] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250905 07:48:13.214375 11177 log.cc:1199] T fac73c227d6746f2b01f416bfa440f5c P 745f84b573b4476190673e98e17c9cb1: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/fac73c227d6746f2b01f416bfa440f5c
I20250905 07:48:13.215142 10720 catalog_manager.cc:4928] TS 745f84b573b4476190673e98e17c9cb1 (127.8.229.65:35271): tablet fac73c227d6746f2b01f416bfa440f5c (table test-workload [id=dab7422ec1eb4cdca85e5112969dcca3]) successfully deleted
I20250905 07:48:14.752813 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:14.753512 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:14.753647 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:16.754159 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:16.754820 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:16.754972 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:18.755473 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:18.756134 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:18.756306 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:20.756827 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:20.757489 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:20.757652 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:22.758149 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:22.758817 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:22.758957 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:24.759460 10789 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:24.760083 10789 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:24.760239 10789 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:25.366103 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:48:25.373968 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:48:25.374233 9109 tablet_replica.cc:331] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1: stopping tablet replica
I20250905 07:48:25.374404 9109 raft_consensus.cc:2241] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:48:25.374718 9109 raft_consensus.cc:2270] T 816721c5807d45968757f82150ee457b P 745f84b573b4476190673e98e17c9cb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:25.391767 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:48:25.402283 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:48:25.410768 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:48:25.411056 9109 tablet_replica.cc:331] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e: stopping tablet replica
I20250905 07:48:25.411259 9109 raft_consensus.cc:2241] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:48:25.411528 9109 raft_consensus.cc:2270] T 816721c5807d45968757f82150ee457b P 4dbb1b1653474259b304dcf018e6a33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:25.412312 9109 tablet_replica.cc:331] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e: stopping tablet replica
I20250905 07:48:25.412467 9109 raft_consensus.cc:2241] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:48:25.412802 9109 raft_consensus.cc:2270] T fac73c227d6746f2b01f416bfa440f5c P 4dbb1b1653474259b304dcf018e6a33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:25.427390 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:48:25.430477 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:48:25.442032 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:48:25.442337 9109 tablet_replica.cc:331] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86: stopping tablet replica
I20250905 07:48:25.442515 9109 raft_consensus.cc:2241] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:48:25.442757 9109 raft_consensus.cc:2270] T fac73c227d6746f2b01f416bfa440f5c P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:25.443284 9109 tablet_replica.cc:331] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86: stopping tablet replica
I20250905 07:48:25.443404 9109 raft_consensus.cc:2241] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:48:25.443563 9109 raft_consensus.cc:2270] T 816721c5807d45968757f82150ee457b P 0eebf4a15ea54cd1886d93620717ea86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:25.456982 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:48:25.460011 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:48:25.467864 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:48:25.468137 9109 tablet_replica.cc:331] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec: stopping tablet replica
I20250905 07:48:25.468303 9109 raft_consensus.cc:2241] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:48:25.468474 9109 raft_consensus.cc:2270] T fac73c227d6746f2b01f416bfa440f5c P 5040468934274f999921dbbe91769dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:25.481805 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:48:25.484624 9109 master.cc:561] Master@127.8.229.126:37117 shutting down...
I20250905 07:48:25.492676 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:48:25.492929 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:25.493065 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 186be7a143ec484e8b15d258a291d2da: stopping tablet replica
I20250905 07:48:25.506454 9109 master.cc:583] Master@127.8.229.126:37117 shutdown complete.
[ OK ] AutoRebalancerTest.MovesScheduledIfAddTserver (26840 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTservers
I20250905 07:48:25.522881 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:39681
I20250905 07:48:25.523401 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:25.525962 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:25.526014 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:25.526091 9109 server_base.cc:1047] running on GCE node
W20250905 07:48:25.526046 11183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:25.526649 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:25.526712 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:25.526736 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058505526733 us; error 0 us; skew 500 ppm
I20250905 07:48:25.526885 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:25.528057 9109 webserver.cc:480] Webserver started at http://127.8.229.126:46055/ using document root <none> and password file <none>
I20250905 07:48:25.528275 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:25.528335 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:25.528474 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:25.529054 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "108fa62bef23405a80d82961be6296b0"
format_stamp: "Formatted at 2025-09-05 07:48:25 on dist-test-slave-pgzt"
I20250905 07:48:25.531487 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:48:25.533149 11188 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:25.533527 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:48:25.533624 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "108fa62bef23405a80d82961be6296b0"
format_stamp: "Formatted at 2025-09-05 07:48:25 on dist-test-slave-pgzt"
I20250905 07:48:25.533731 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:25.570895 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:25.571480 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:25.582861 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:39681
I20250905 07:48:25.582891 11240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:39681 every 8 connection(s)
I20250905 07:48:25.584303 11241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:25.588235 11241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0: Bootstrap starting.
I20250905 07:48:25.589325 11241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:25.590696 11241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0: No bootstrap required, opened a new log
I20250905 07:48:25.591264 11241 raft_consensus.cc:357] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108fa62bef23405a80d82961be6296b0" member_type: VOTER }
I20250905 07:48:25.591384 11241 raft_consensus.cc:383] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:25.591470 11241 raft_consensus.cc:738] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108fa62bef23405a80d82961be6296b0, State: Initialized, Role: FOLLOWER
I20250905 07:48:25.591660 11241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108fa62bef23405a80d82961be6296b0" member_type: VOTER }
I20250905 07:48:25.591755 11241 raft_consensus.cc:397] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:48:25.591812 11241 raft_consensus.cc:491] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:48:25.591883 11241 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:25.593562 11241 raft_consensus.cc:513] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108fa62bef23405a80d82961be6296b0" member_type: VOTER }
I20250905 07:48:25.593715 11241 leader_election.cc:304] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 108fa62bef23405a80d82961be6296b0; no voters:
I20250905 07:48:25.594085 11241 leader_election.cc:290] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:48:25.594182 11244 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:48:25.594604 11244 raft_consensus.cc:695] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 1 LEADER]: Becoming Leader. State: Replica: 108fa62bef23405a80d82961be6296b0, State: Running, Role: LEADER
I20250905 07:48:25.594839 11244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108fa62bef23405a80d82961be6296b0" member_type: VOTER }
I20250905 07:48:25.594897 11241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:48:25.595559 11246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 108fa62bef23405a80d82961be6296b0. Latest consensus state: current_term: 1 leader_uuid: "108fa62bef23405a80d82961be6296b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108fa62bef23405a80d82961be6296b0" member_type: VOTER } }
I20250905 07:48:25.595633 11245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "108fa62bef23405a80d82961be6296b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108fa62bef23405a80d82961be6296b0" member_type: VOTER } }
I20250905 07:48:25.595844 11245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [sys.catalog]: This master's current role is: LEADER
I20250905 07:48:25.595732 11246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [sys.catalog]: This master's current role is: LEADER
I20250905 07:48:25.597062 11250 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:48:25.598956 11250 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:48:25.599634 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:48:25.601828 11250 catalog_manager.cc:1349] Generated new cluster ID: b565d2332c174716b7372094c9fe01d1
I20250905 07:48:25.601897 11250 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:48:25.619279 11250 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:48:25.620097 11250 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:48:25.629889 11250 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0: Generated new TSK 0
I20250905 07:48:25.630146 11250 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:48:25.631708 9109 internal_mini_cluster.cc:371] 0 TS(s) registered with all masters after 6.3807e-05s
I20250905 07:48:37.713536 9109 master.cc:561] Master@127.8.229.126:39681 shutting down...
I20250905 07:48:37.729096 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:48:37.729348 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:37.729457 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 108fa62bef23405a80d82961be6296b0: stopping tablet replica
I20250905 07:48:37.743413 9109 master.cc:583] Master@127.8.229.126:39681 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTservers (12228 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTablets
I20250905 07:48:37.751341 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:37113
I20250905 07:48:37.751884 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:37.754215 11263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.754539 11264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.755013 11266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:37.758746 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:37.760071 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:37.760144 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:37.760180 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058517760166 us; error 0 us; skew 500 ppm
I20250905 07:48:37.760316 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:37.761423 9109 webserver.cc:480] Webserver started at http://127.8.229.126:35579/ using document root <none> and password file <none>
I20250905 07:48:37.761615 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:37.761680 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:37.761790 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:37.762375 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "593cbacb8e084f1ea6f4ebd89975da56"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.764744 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:48:37.766103 11271 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.766448 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:48:37.766556 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "593cbacb8e084f1ea6f4ebd89975da56"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.766680 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:37.773304 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:37.773742 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:37.783631 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:37113
I20250905 07:48:37.783658 11323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:37113 every 8 connection(s)
I20250905 07:48:37.784865 11324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:37.788951 11324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56: Bootstrap starting.
I20250905 07:48:37.790012 11324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:37.791358 11324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56: No bootstrap required, opened a new log
I20250905 07:48:37.791918 11324 raft_consensus.cc:357] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593cbacb8e084f1ea6f4ebd89975da56" member_type: VOTER }
I20250905 07:48:37.792058 11324 raft_consensus.cc:383] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:37.792124 11324 raft_consensus.cc:738] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593cbacb8e084f1ea6f4ebd89975da56, State: Initialized, Role: FOLLOWER
I20250905 07:48:37.792320 11324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593cbacb8e084f1ea6f4ebd89975da56" member_type: VOTER }
I20250905 07:48:37.792443 11324 raft_consensus.cc:397] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:48:37.792501 11324 raft_consensus.cc:491] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:48:37.792569 11324 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:37.794207 11324 raft_consensus.cc:513] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593cbacb8e084f1ea6f4ebd89975da56" member_type: VOTER }
I20250905 07:48:37.794391 11324 leader_election.cc:304] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 593cbacb8e084f1ea6f4ebd89975da56; no voters:
I20250905 07:48:37.794713 11324 leader_election.cc:290] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:48:37.794826 11327 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:48:37.795284 11327 raft_consensus.cc:695] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 1 LEADER]: Becoming Leader. State: Replica: 593cbacb8e084f1ea6f4ebd89975da56, State: Running, Role: LEADER
I20250905 07:48:37.795583 11324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:48:37.795562 11327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593cbacb8e084f1ea6f4ebd89975da56" member_type: VOTER }
I20250905 07:48:37.796384 11328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "593cbacb8e084f1ea6f4ebd89975da56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593cbacb8e084f1ea6f4ebd89975da56" member_type: VOTER } }
I20250905 07:48:37.796419 11329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 593cbacb8e084f1ea6f4ebd89975da56. Latest consensus state: current_term: 1 leader_uuid: "593cbacb8e084f1ea6f4ebd89975da56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593cbacb8e084f1ea6f4ebd89975da56" member_type: VOTER } }
I20250905 07:48:37.796631 11328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [sys.catalog]: This master's current role is: LEADER
I20250905 07:48:37.796694 11329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [sys.catalog]: This master's current role is: LEADER
I20250905 07:48:37.797797 11332 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:48:37.799245 11332 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:48:37.800976 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:48:37.802134 11332 catalog_manager.cc:1349] Generated new cluster ID: c31c35c4f8984603956684d2ad6fb8f9
I20250905 07:48:37.802248 11332 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:48:37.816598 11332 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:48:37.817358 11332 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:48:37.826619 11332 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56: Generated new TSK 0
I20250905 07:48:37.826831 11332 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:48:37.833573 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:37.836146 11345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.836555 11346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.836808 11348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:37.837373 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:37.837597 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:37.837661 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:37.837703 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058517837689 us; error 0 us; skew 500 ppm
I20250905 07:48:37.837819 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:37.838925 9109 webserver.cc:480] Webserver started at http://127.8.229.65:33065/ using document root <none> and password file <none>
I20250905 07:48:37.839129 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:37.839226 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:37.839339 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:37.839987 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "e894ae16ede14614bee534f2ae708379"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.842345 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20250905 07:48:37.843827 11353 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.844255 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:48:37.844369 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "e894ae16ede14614bee534f2ae708379"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.844513 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:37.858981 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:37.859597 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:37.860162 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:37.860998 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:37.861075 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.861195 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:37.861266 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.874050 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:36969
I20250905 07:48:37.874081 11415 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:36969 every 8 connection(s)
I20250905 07:48:37.875658 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:37.878525 11420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.879027 11421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.879309 11423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:37.882367 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:37.882591 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:37.882658 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:37.882722 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058517882704 us; error 0 us; skew 500 ppm
I20250905 07:48:37.882885 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:37.883982 9109 webserver.cc:480] Webserver started at http://127.8.229.66:37315/ using document root <none> and password file <none>
I20250905 07:48:37.884220 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:37.884300 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:37.884442 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:37.884999 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "c4e36bb51801424fb2b84b9436d63a1a"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.887535 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:48:37.888264 11416 heartbeater.cc:344] Connected to a master server at 127.8.229.126:37113
I20250905 07:48:37.888414 11416 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:37.888754 11416 heartbeater.cc:507] Master 127.8.229.126:37113 requested a full tablet report, sending...
I20250905 07:48:37.889416 11428 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.889766 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:48:37.889827 11288 ts_manager.cc:194] Registered new tserver with Master: e894ae16ede14614bee534f2ae708379 (127.8.229.65:36969)
I20250905 07:48:37.889882 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "c4e36bb51801424fb2b84b9436d63a1a"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.890008 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:37.891052 11288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37194
I20250905 07:48:37.900686 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:37.901268 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:37.901769 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:37.902522 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:37.902594 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.902695 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:37.902751 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.914557 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:34273
I20250905 07:48:37.914582 11491 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:34273 every 8 connection(s)
I20250905 07:48:37.916028 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:37.918622 11495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.919051 11496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:37.920423 11498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:37.920444 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:37.920912 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:37.921012 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:37.921075 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058517921051 us; error 0 us; skew 500 ppm
I20250905 07:48:37.921267 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:37.922688 9109 webserver.cc:480] Webserver started at http://127.8.229.67:35041/ using document root <none> and password file <none>
I20250905 07:48:37.922967 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:37.923063 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:37.923244 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:37.923681 11492 heartbeater.cc:344] Connected to a master server at 127.8.229.126:37113
I20250905 07:48:37.923804 11492 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:37.923921 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "a8791a6025cc489e80bd0c18126781a9"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.924134 11492 heartbeater.cc:507] Master 127.8.229.126:37113 requested a full tablet report, sending...
I20250905 07:48:37.924963 11288 ts_manager.cc:194] Registered new tserver with Master: c4e36bb51801424fb2b84b9436d63a1a (127.8.229.66:34273)
I20250905 07:48:37.925635 11288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37198
I20250905 07:48:37.926383 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:48:37.927917 11503 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.928278 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:48:37.928393 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "a8791a6025cc489e80bd0c18126781a9"
format_stamp: "Formatted at 2025-09-05 07:48:37 on dist-test-slave-pgzt"
I20250905 07:48:37.928503 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:37.935036 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:37.935544 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:37.935985 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:37.936808 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:37.936895 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.936988 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:37.937049 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:37.948745 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:38859
I20250905 07:48:37.948779 11566 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:38859 every 8 connection(s)
I20250905 07:48:37.955374 11567 heartbeater.cc:344] Connected to a master server at 127.8.229.126:37113
I20250905 07:48:37.955546 11567 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:37.955868 11567 heartbeater.cc:507] Master 127.8.229.126:37113 requested a full tablet report, sending...
I20250905 07:48:37.956681 11288 ts_manager.cc:194] Registered new tserver with Master: a8791a6025cc489e80bd0c18126781a9 (127.8.229.67:38859)
I20250905 07:48:37.957382 11288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37214
I20250905 07:48:37.957554 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007889707s
I20250905 07:48:38.892593 11416 heartbeater.cc:499] Master 127.8.229.126:37113 was elected leader, sending a full tablet report...
I20250905 07:48:38.927125 11492 heartbeater.cc:499] Master 127.8.229.126:37113 was elected leader, sending a full tablet report...
I20250905 07:48:38.958671 11567 heartbeater.cc:499] Master 127.8.229.126:37113 was elected leader, sending a full tablet report...
I20250905 07:48:39.800911 11342 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:41.801328 11342 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:43.801813 11342 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:45.802307 11342 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:47.802834 11342 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:49.803392 11342 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:50.039788 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:48:50.053035 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:48:50.066180 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:48:50.075927 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:48:50.083132 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:48:50.095860 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:48:50.098678 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:48:50.106783 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:48:50.119724 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:48:50.122480 9109 master.cc:561] Master@127.8.229.126:37113 shutting down...
I20250905 07:48:50.129889 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:48:50.130121 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:48:50.130223 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 593cbacb8e084f1ea6f4ebd89975da56: stopping tablet replica
I20250905 07:48:50.143834 9109 master.cc:583] Master@127.8.229.126:37113 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTablets (12401 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne
I20250905 07:48:50.152952 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:33329
I20250905 07:48:50.153487 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:50.156069 11572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.156081 11571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.156165 11574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:50.156504 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:50.156829 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:50.156881 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:50.156903 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058530156900 us; error 0 us; skew 500 ppm
I20250905 07:48:50.157011 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:50.158150 9109 webserver.cc:480] Webserver started at http://127.8.229.126:39023/ using document root <none> and password file <none>
I20250905 07:48:50.158334 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:50.158386 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:50.158480 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:50.159056 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "85b1bb41b5d04104bf55b1810626e5f0"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.162132 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250905 07:48:50.164338 11579 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.164672 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:48:50.164768 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "85b1bb41b5d04104bf55b1810626e5f0"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.164875 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:50.181217 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:50.181816 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:50.194213 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:33329
I20250905 07:48:50.194252 11631 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:33329 every 8 connection(s)
I20250905 07:48:50.195473 11632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:50.199280 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0: Bootstrap starting.
I20250905 07:48:50.200618 11632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:50.202088 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0: No bootstrap required, opened a new log
I20250905 07:48:50.202682 11632 raft_consensus.cc:357] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85b1bb41b5d04104bf55b1810626e5f0" member_type: VOTER }
I20250905 07:48:50.202797 11632 raft_consensus.cc:383] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:50.202833 11632 raft_consensus.cc:738] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85b1bb41b5d04104bf55b1810626e5f0, State: Initialized, Role: FOLLOWER
I20250905 07:48:50.203028 11632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85b1bb41b5d04104bf55b1810626e5f0" member_type: VOTER }
I20250905 07:48:50.203136 11632 raft_consensus.cc:397] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:48:50.203199 11632 raft_consensus.cc:491] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:48:50.203276 11632 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:50.204909 11632 raft_consensus.cc:513] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85b1bb41b5d04104bf55b1810626e5f0" member_type: VOTER }
I20250905 07:48:50.205071 11632 leader_election.cc:304] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85b1bb41b5d04104bf55b1810626e5f0; no voters:
I20250905 07:48:50.205425 11632 leader_election.cc:290] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:48:50.205533 11635 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:48:50.206009 11635 raft_consensus.cc:695] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 1 LEADER]: Becoming Leader. State: Replica: 85b1bb41b5d04104bf55b1810626e5f0, State: Running, Role: LEADER
I20250905 07:48:50.206269 11635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85b1bb41b5d04104bf55b1810626e5f0" member_type: VOTER }
I20250905 07:48:50.206272 11632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:48:50.207114 11636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85b1bb41b5d04104bf55b1810626e5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85b1bb41b5d04104bf55b1810626e5f0" member_type: VOTER } }
I20250905 07:48:50.207166 11637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85b1bb41b5d04104bf55b1810626e5f0. Latest consensus state: current_term: 1 leader_uuid: "85b1bb41b5d04104bf55b1810626e5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85b1bb41b5d04104bf55b1810626e5f0" member_type: VOTER } }
I20250905 07:48:50.207360 11636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [sys.catalog]: This master's current role is: LEADER
I20250905 07:48:50.207474 11637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [sys.catalog]: This master's current role is: LEADER
I20250905 07:48:50.211349 11640 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:48:50.211568 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:48:50.212659 11640 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:48:50.215199 11640 catalog_manager.cc:1349] Generated new cluster ID: 7b399c1dbb9a467faa9a225139ebda3f
I20250905 07:48:50.215276 11640 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:48:50.245808 11640 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:48:50.246564 11640 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:48:50.252565 11640 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0: Generated new TSK 0
I20250905 07:48:50.252803 11640 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:48:50.276579 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:50.279481 11654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.279541 11653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.279760 11656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:50.279930 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:50.280213 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:50.280272 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:50.280295 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058530280292 us; error 0 us; skew 500 ppm
I20250905 07:48:50.280405 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:50.281508 9109 webserver.cc:480] Webserver started at http://127.8.229.65:38863/ using document root <none> and password file <none>
I20250905 07:48:50.281689 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:50.281746 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:50.281844 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:50.282426 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "267f3eec6c2441eb8084723dcd2dc345"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.284822 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:48:50.286190 11661 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.286500 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:48:50.286589 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "267f3eec6c2441eb8084723dcd2dc345"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.286695 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:50.301146 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:50.301717 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:50.302295 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:50.303294 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:50.303360 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.303452 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:50.303498 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.320770 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:42389
I20250905 07:48:50.320791 11723 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:42389 every 8 connection(s)
I20250905 07:48:50.322492 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:50.325632 11728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.325649 11729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.327090 11731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:50.328475 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:50.328739 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:50.328795 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:50.328843 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058530328838 us; error 0 us; skew 500 ppm
I20250905 07:48:50.328980 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:50.330204 9109 webserver.cc:480] Webserver started at http://127.8.229.66:37887/ using document root <none> and password file <none>
I20250905 07:48:50.330404 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:50.330480 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:50.330610 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:50.331236 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "ac9b860c91be4634a084e24e2009acc9"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.332576 11724 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33329
I20250905 07:48:50.332697 11724 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:50.333016 11724 heartbeater.cc:507] Master 127.8.229.126:33329 requested a full tablet report, sending...
I20250905 07:48:50.333906 11596 ts_manager.cc:194] Registered new tserver with Master: 267f3eec6c2441eb8084723dcd2dc345 (127.8.229.65:42389)
I20250905 07:48:50.334048 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:48:50.335413 11596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33400
I20250905 07:48:50.335912 11736 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.336773 9109 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250905 07:48:50.336859 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "ac9b860c91be4634a084e24e2009acc9"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.336942 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:50.347278 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:50.347908 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:50.348529 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:50.349334 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:50.349396 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.349463 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:50.349499 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.361166 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:37429
I20250905 07:48:50.361193 11798 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:37429 every 8 connection(s)
I20250905 07:48:50.362663 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:50.365057 11802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.367460 11803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.367904 11805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:50.368187 9109 server_base.cc:1047] running on GCE node
I20250905 07:48:50.368492 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:50.368552 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:50.368587 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058530368582 us; error 0 us; skew 500 ppm
I20250905 07:48:50.368747 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:50.369979 9109 webserver.cc:480] Webserver started at http://127.8.229.67:39923/ using document root <none> and password file <none>
I20250905 07:48:50.370205 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:50.370289 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:50.370400 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:50.370435 11799 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33329
I20250905 07:48:50.370560 11799 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:50.370918 11799 heartbeater.cc:507] Master 127.8.229.126:33329 requested a full tablet report, sending...
I20250905 07:48:50.371127 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "96e052d375164ab5ba9db246a8d16f2e"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.371939 11596 ts_manager.cc:194] Registered new tserver with Master: ac9b860c91be4634a084e24e2009acc9 (127.8.229.66:37429)
I20250905 07:48:50.372977 11596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33412
I20250905 07:48:50.375170 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20250905 07:48:50.376745 11811 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.377058 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:48:50.377168 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "96e052d375164ab5ba9db246a8d16f2e"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.377264 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:50.390561 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:50.391085 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:50.391646 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:50.392436 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:50.392489 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.392572 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:50.392616 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.404426 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:45191
I20250905 07:48:50.404445 11873 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:45191 every 8 connection(s)
I20250905 07:48:50.405965 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:48:50.408710 11877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:48:50.408754 11878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:50.410256 9109 server_base.cc:1047] running on GCE node
W20250905 07:48:50.411590 11880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:48:50.412547 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:48:50.412627 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:48:50.412669 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058530412665 us; error 0 us; skew 500 ppm
I20250905 07:48:50.412699 11874 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33329
I20250905 07:48:50.412779 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:48:50.412801 11874 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:50.413165 11874 heartbeater.cc:507] Master 127.8.229.126:33329 requested a full tablet report, sending...
I20250905 07:48:50.414012 9109 webserver.cc:480] Webserver started at http://127.8.229.68:43313/ using document root <none> and password file <none>
I20250905 07:48:50.414180 11596 ts_manager.cc:194] Registered new tserver with Master: 96e052d375164ab5ba9db246a8d16f2e (127.8.229.67:45191)
I20250905 07:48:50.414227 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:48:50.414285 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:48:50.414398 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:48:50.414942 11596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33422
I20250905 07:48:50.414978 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "a4dfb7f5fb984e02b0ce0a540297f881"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.417397 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:48:50.418814 11885 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.419140 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:48:50.419216 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "a4dfb7f5fb984e02b0ce0a540297f881"
format_stamp: "Formatted at 2025-09-05 07:48:50 on dist-test-slave-pgzt"
I20250905 07:48:50.419327 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:48:50.429100 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:48:50.429798 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:48:50.430490 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:48:50.431612 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:48:50.431686 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.431771 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:48:50.431825 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:48:50.449249 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:39969
I20250905 07:48:50.449265 11948 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:39969 every 8 connection(s)
I20250905 07:48:50.463835 11949 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33329
I20250905 07:48:50.464007 11949 heartbeater.cc:461] Registering TS with master...
I20250905 07:48:50.464434 11949 heartbeater.cc:507] Master 127.8.229.126:33329 requested a full tablet report, sending...
I20250905 07:48:50.465327 11596 ts_manager.cc:194] Registered new tserver with Master: a4dfb7f5fb984e02b0ce0a540297f881 (127.8.229.68:39969)
I20250905 07:48:50.466166 11596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33432
I20250905 07:48:50.466384 9109 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.01573579s
I20250905 07:48:51.336772 11724 heartbeater.cc:499] Master 127.8.229.126:33329 was elected leader, sending a full tablet report...
I20250905 07:48:51.374457 11799 heartbeater.cc:499] Master 127.8.229.126:33329 was elected leader, sending a full tablet report...
I20250905 07:48:51.416157 11874 heartbeater.cc:499] Master 127.8.229.126:33329 was elected leader, sending a full tablet report...
I20250905 07:48:51.467831 11949 heartbeater.cc:499] Master 127.8.229.126:33329 was elected leader, sending a full tablet report...
I20250905 07:48:51.501220 9109 test_util.cc:276] Using random seed: 295609502
I20250905 07:48:51.513787 11596 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45986:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250905 07:48:51.546636 11914 tablet_service.cc:1468] Processing CreateTablet for tablet 233cfae074f049259170576146d8ddfa (DEFAULT_TABLE table=test-workload [id=14f57d9220274f339fed4cdf081e8a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:48:51.547184 11914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233cfae074f049259170576146d8ddfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:51.547113 11764 tablet_service.cc:1468] Processing CreateTablet for tablet 233cfae074f049259170576146d8ddfa (DEFAULT_TABLE table=test-workload [id=14f57d9220274f339fed4cdf081e8a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:48:51.547370 11839 tablet_service.cc:1468] Processing CreateTablet for tablet 233cfae074f049259170576146d8ddfa (DEFAULT_TABLE table=test-workload [id=14f57d9220274f339fed4cdf081e8a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:48:51.547540 11764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233cfae074f049259170576146d8ddfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:51.547796 11839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233cfae074f049259170576146d8ddfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:48:51.554207 11970 tablet_bootstrap.cc:492] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e: Bootstrap starting.
I20250905 07:48:51.555873 11970 tablet_bootstrap.cc:654] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:51.557370 11971 tablet_bootstrap.cc:492] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9: Bootstrap starting.
I20250905 07:48:51.557886 11970 tablet_bootstrap.cc:492] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e: No bootstrap required, opened a new log
I20250905 07:48:51.558009 11970 ts_tablet_manager.cc:1397] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:48:51.558874 11970 raft_consensus.cc:357] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.559103 11970 raft_consensus.cc:383] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:51.559171 11970 raft_consensus.cc:738] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e052d375164ab5ba9db246a8d16f2e, State: Initialized, Role: FOLLOWER
I20250905 07:48:51.559525 11972 tablet_bootstrap.cc:492] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881: Bootstrap starting.
I20250905 07:48:51.559656 11970 consensus_queue.cc:260] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.560236 11971 tablet_bootstrap.cc:654] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:51.560616 11970 ts_tablet_manager.cc:1428] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20250905 07:48:51.561659 11972 tablet_bootstrap.cc:654] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881: Neither blocks nor log segments found. Creating new log.
I20250905 07:48:51.563323 11971 tablet_bootstrap.cc:492] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9: No bootstrap required, opened a new log
I20250905 07:48:51.563408 11972 tablet_bootstrap.cc:492] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881: No bootstrap required, opened a new log
I20250905 07:48:51.563508 11971 ts_tablet_manager.cc:1397] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250905 07:48:51.563580 11972 ts_tablet_manager.cc:1397] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20250905 07:48:51.564174 11971 raft_consensus.cc:357] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.564307 11971 raft_consensus.cc:383] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:51.564385 11971 raft_consensus.cc:738] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9b860c91be4634a084e24e2009acc9, State: Initialized, Role: FOLLOWER
I20250905 07:48:51.564349 11972 raft_consensus.cc:357] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.564589 11971 consensus_queue.cc:260] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.564630 11972 raft_consensus.cc:383] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:48:51.564889 11972 raft_consensus.cc:738] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4dfb7f5fb984e02b0ce0a540297f881, State: Initialized, Role: FOLLOWER
I20250905 07:48:51.565135 11972 consensus_queue.cc:260] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.566144 11971 ts_tablet_manager.cc:1428] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:48:51.566601 11972 ts_tablet_manager.cc:1428] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20250905 07:48:51.613139 11800 tablet.cc:2378] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250905 07:48:51.665890 11977 raft_consensus.cc:491] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:48:51.666061 11977 raft_consensus.cc:513] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.666913 11977 leader_election.cc:290] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4dfb7f5fb984e02b0ce0a540297f881 (127.8.229.68:39969), 96e052d375164ab5ba9db246a8d16f2e (127.8.229.67:45191)
I20250905 07:48:51.676157 11924 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "233cfae074f049259170576146d8ddfa" candidate_uuid: "ac9b860c91be4634a084e24e2009acc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" is_pre_election: true
I20250905 07:48:51.676465 11924 raft_consensus.cc:2466] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac9b860c91be4634a084e24e2009acc9 in term 0.
I20250905 07:48:51.676537 11849 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "233cfae074f049259170576146d8ddfa" candidate_uuid: "ac9b860c91be4634a084e24e2009acc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e052d375164ab5ba9db246a8d16f2e" is_pre_election: true
I20250905 07:48:51.676791 11849 raft_consensus.cc:2466] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac9b860c91be4634a084e24e2009acc9 in term 0.
I20250905 07:48:51.676930 11738 leader_election.cc:304] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4dfb7f5fb984e02b0ce0a540297f881, ac9b860c91be4634a084e24e2009acc9; no voters:
I20250905 07:48:51.677207 11977 raft_consensus.cc:2802] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:48:51.677302 11977 raft_consensus.cc:491] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:48:51.677376 11977 raft_consensus.cc:3058] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:51.680195 11977 raft_consensus.cc:513] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.680671 11977 leader_election.cc:290] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [CANDIDATE]: Term 1 election: Requested vote from peers a4dfb7f5fb984e02b0ce0a540297f881 (127.8.229.68:39969), 96e052d375164ab5ba9db246a8d16f2e (127.8.229.67:45191)
I20250905 07:48:51.681100 11849 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "233cfae074f049259170576146d8ddfa" candidate_uuid: "ac9b860c91be4634a084e24e2009acc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e052d375164ab5ba9db246a8d16f2e"
I20250905 07:48:51.681097 11924 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "233cfae074f049259170576146d8ddfa" candidate_uuid: "ac9b860c91be4634a084e24e2009acc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4dfb7f5fb984e02b0ce0a540297f881"
I20250905 07:48:51.681242 11849 raft_consensus.cc:3058] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:51.681324 11924 raft_consensus.cc:3058] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:48:51.683218 11849 raft_consensus.cc:2466] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac9b860c91be4634a084e24e2009acc9 in term 1.
I20250905 07:48:51.683425 11924 raft_consensus.cc:2466] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac9b860c91be4634a084e24e2009acc9 in term 1.
I20250905 07:48:51.683581 11740 leader_election.cc:304] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e052d375164ab5ba9db246a8d16f2e, ac9b860c91be4634a084e24e2009acc9; no voters:
I20250905 07:48:51.683928 11977 raft_consensus.cc:2802] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:48:51.684290 11977 raft_consensus.cc:695] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 1 LEADER]: Becoming Leader. State: Replica: ac9b860c91be4634a084e24e2009acc9, State: Running, Role: LEADER
I20250905 07:48:51.684520 11977 consensus_queue.cc:237] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } }
I20250905 07:48:51.686789 11596 catalog_manager.cc:5582] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac9b860c91be4634a084e24e2009acc9 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "ac9b860c91be4634a084e24e2009acc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac9b860c91be4634a084e24e2009acc9" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 37429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 } health_report { overall_health: UNKNOWN } } }
I20250905 07:48:52.164701 11977 consensus_queue.cc:1035] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4dfb7f5fb984e02b0ce0a540297f881" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:48:52.169289 11977 consensus_queue.cc:1035] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e052d375164ab5ba9db246a8d16f2e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:48:52.211136 11650 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:52.211642 11650 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:52.211756 11650 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:54.212121 11650 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:54.212519 11650 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:54.212607 11650 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:56.212949 11650 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:56.213408 11650 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:56.213495 11650 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:48:58.213861 11650 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:48:58.214347 11650 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:48:58.214473 11650 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:00.214869 11650 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:00.215320 11650 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:00.215473 11650 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:02.215931 11650 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:02.216411 11650 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:02.216521 11650 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:02.808576 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:49:02.818997 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:02.834957 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:49:02.837677 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:49:02.846275 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:02.846510 9109 tablet_replica.cc:331] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9: stopping tablet replica
I20250905 07:49:02.846697 9109 raft_consensus.cc:2241] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:49:02.847019 9109 raft_consensus.cc:2270] T 233cfae074f049259170576146d8ddfa P ac9b860c91be4634a084e24e2009acc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:02.860677 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:49:02.863492 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:49:02.872990 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:02.873250 9109 tablet_replica.cc:331] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e: stopping tablet replica
I20250905 07:49:02.873411 9109 raft_consensus.cc:2241] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:02.873607 9109 raft_consensus.cc:2270] T 233cfae074f049259170576146d8ddfa P 96e052d375164ab5ba9db246a8d16f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:02.888356 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:49:02.891144 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:49:02.900799 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:02.901038 9109 tablet_replica.cc:331] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881: stopping tablet replica
I20250905 07:49:02.901233 9109 raft_consensus.cc:2241] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:02.901407 9109 raft_consensus.cc:2270] T 233cfae074f049259170576146d8ddfa P a4dfb7f5fb984e02b0ce0a540297f881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:02.915009 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:49:02.917905 9109 master.cc:561] Master@127.8.229.126:33329 shutting down...
I20250905 07:49:02.925752 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:49:02.926009 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:02.926136 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 85b1bb41b5d04104bf55b1810626e5f0: stopping tablet replica
I20250905 07:49:02.939603 9109 master.cc:583] Master@127.8.229.126:33329 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne (12808 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy
I20250905 07:49:02.961236 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:40237
I20250905 07:49:02.961777 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:02.964303 11994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:02.964529 11997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:02.964550 11995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:02.964779 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:02.965068 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:02.965127 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:02.965165 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058542965159 us; error 0 us; skew 500 ppm
I20250905 07:49:02.965273 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:02.966374 9109 webserver.cc:480] Webserver started at http://127.8.229.126:40367/ using document root <none> and password file <none>
I20250905 07:49:02.966555 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:02.966604 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:02.966703 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:02.967353 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "0904f9d5b8744c33b0c77ce212c1aaf5"
format_stamp: "Formatted at 2025-09-05 07:49:02 on dist-test-slave-pgzt"
I20250905 07:49:02.969821 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20250905 07:49:02.971307 12002 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:02.971653 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:49:02.971755 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "0904f9d5b8744c33b0c77ce212c1aaf5"
format_stamp: "Formatted at 2025-09-05 07:49:02 on dist-test-slave-pgzt"
I20250905 07:49:02.971863 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:02.981889 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:02.982350 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:02.992555 12053 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:40237 every 8 connection(s)
I20250905 07:49:02.992535 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:40237
I20250905 07:49:02.993778 12054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:02.997763 12054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5: Bootstrap starting.
I20250905 07:49:02.998922 12054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:03.000411 12054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5: No bootstrap required, opened a new log
I20250905 07:49:03.000970 12054 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" member_type: VOTER }
I20250905 07:49:03.001118 12054 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:03.001174 12054 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0904f9d5b8744c33b0c77ce212c1aaf5, State: Initialized, Role: FOLLOWER
I20250905 07:49:03.001371 12054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" member_type: VOTER }
I20250905 07:49:03.001470 12054 raft_consensus.cc:397] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:49:03.001526 12054 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:49:03.001600 12054 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:03.003225 12054 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" member_type: VOTER }
I20250905 07:49:03.003378 12054 leader_election.cc:304] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0904f9d5b8744c33b0c77ce212c1aaf5; no voters:
I20250905 07:49:03.003800 12054 leader_election.cc:290] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:49:03.003926 12057 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:03.004406 12057 raft_consensus.cc:695] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 1 LEADER]: Becoming Leader. State: Replica: 0904f9d5b8744c33b0c77ce212c1aaf5, State: Running, Role: LEADER
I20250905 07:49:03.004599 12054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:49:03.004609 12057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" member_type: VOTER }
I20250905 07:49:03.005854 12059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0904f9d5b8744c33b0c77ce212c1aaf5. Latest consensus state: current_term: 1 leader_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" member_type: VOTER } }
I20250905 07:49:03.006029 12059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [sys.catalog]: This master's current role is: LEADER
I20250905 07:49:03.006630 12065 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:49:03.006665 12058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0904f9d5b8744c33b0c77ce212c1aaf5" member_type: VOTER } }
I20250905 07:49:03.006824 12058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [sys.catalog]: This master's current role is: LEADER
I20250905 07:49:03.008122 12065 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:49:03.009838 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:49:03.010860 12065 catalog_manager.cc:1349] Generated new cluster ID: f7041dd20d90482e9c0096e139a33af6
I20250905 07:49:03.010931 12065 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:49:03.042029 12065 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:49:03.042760 12065 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:49:03.048298 12065 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5: Generated new TSK 0
I20250905 07:49:03.048499 12065 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:49:03.074716 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:03.077364 12075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:03.077481 12076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:03.077939 12078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:03.078291 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:03.078517 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:03.078568 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:03.078604 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058543078601 us; error 0 us; skew 500 ppm
I20250905 07:49:03.078745 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:03.079978 9109 webserver.cc:480] Webserver started at http://127.8.229.65:38355/ using document root <none> and password file <none>
I20250905 07:49:03.080199 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:03.080266 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:03.080401 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:03.081035 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "aa000456317c43d88988041c854b91de"
format_stamp: "Formatted at 2025-09-05 07:49:03 on dist-test-slave-pgzt"
I20250905 07:49:03.083421 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20250905 07:49:03.084985 12083 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.085379 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:03.085464 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "aa000456317c43d88988041c854b91de"
format_stamp: "Formatted at 2025-09-05 07:49:03 on dist-test-slave-pgzt"
I20250905 07:49:03.085549 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:03.101220 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:03.101722 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:03.102252 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:03.103129 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:03.103204 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.103272 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:03.103313 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.115595 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:33521
I20250905 07:49:03.115621 12145 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:33521 every 8 connection(s)
I20250905 07:49:03.117223 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:03.120129 12150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:03.121665 12153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:03.121920 12151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:03.123077 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:03.123710 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:03.123780 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:03.123811 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058543123808 us; error 0 us; skew 500 ppm
I20250905 07:49:03.123950 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:03.125074 12146 heartbeater.cc:344] Connected to a master server at 127.8.229.126:40237
I20250905 07:49:03.125204 12146 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:03.125368 9109 webserver.cc:480] Webserver started at http://127.8.229.66:39227/ using document root <none> and password file <none>
I20250905 07:49:03.125569 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:03.125639 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:03.125614 12146 heartbeater.cc:507] Master 127.8.229.126:40237 requested a full tablet report, sending...
I20250905 07:49:03.125767 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:03.126394 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "763ae09a04684fb2a6f15af0d5e38401"
format_stamp: "Formatted at 2025-09-05 07:49:03 on dist-test-slave-pgzt"
I20250905 07:49:03.126487 12019 ts_manager.cc:194] Registered new tserver with Master: aa000456317c43d88988041c854b91de (127.8.229.65:33521)
I20250905 07:49:03.127553 12019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48756
I20250905 07:49:03.128882 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:49:03.130411 12158 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.130769 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:49:03.130867 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "763ae09a04684fb2a6f15af0d5e38401"
format_stamp: "Formatted at 2025-09-05 07:49:03 on dist-test-slave-pgzt"
I20250905 07:49:03.130980 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:03.164988 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:03.165488 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:03.166033 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:03.166880 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:03.166937 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.167011 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:03.167055 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.178951 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:36761
I20250905 07:49:03.178977 12220 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:36761 every 8 connection(s)
I20250905 07:49:03.180516 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:03.183349 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:03.183357 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:03.184187 12228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:03.184667 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:03.184974 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:03.185060 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:03.185494 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058543185486 us; error 0 us; skew 500 ppm
I20250905 07:49:03.185672 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:03.186997 12221 heartbeater.cc:344] Connected to a master server at 127.8.229.126:40237
I20250905 07:49:03.187075 9109 webserver.cc:480] Webserver started at http://127.8.229.67:32957/ using document root <none> and password file <none>
I20250905 07:49:03.187127 12221 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:03.187319 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:03.187389 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:03.187507 12221 heartbeater.cc:507] Master 127.8.229.126:40237 requested a full tablet report, sending...
I20250905 07:49:03.187525 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:03.188304 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "e24ec9dd61fa4f3494c0dd473742c4b9"
format_stamp: "Formatted at 2025-09-05 07:49:03 on dist-test-slave-pgzt"
I20250905 07:49:03.188424 12019 ts_manager.cc:194] Registered new tserver with Master: 763ae09a04684fb2a6f15af0d5e38401 (127.8.229.66:36761)
I20250905 07:49:03.189477 12019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48760
I20250905 07:49:03.191088 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:49:03.192687 12232 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.193022 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:03.193151 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "e24ec9dd61fa4f3494c0dd473742c4b9"
format_stamp: "Formatted at 2025-09-05 07:49:03 on dist-test-slave-pgzt"
I20250905 07:49:03.193266 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:03.202351 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:03.202826 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:03.203320 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:03.204144 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:03.204218 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.204327 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:03.204389 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:03.215075 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:37733
I20250905 07:49:03.215111 12294 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:37733 every 8 connection(s)
I20250905 07:49:03.221333 12295 heartbeater.cc:344] Connected to a master server at 127.8.229.126:40237
I20250905 07:49:03.221474 12295 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:03.221772 12295 heartbeater.cc:507] Master 127.8.229.126:40237 requested a full tablet report, sending...
I20250905 07:49:03.222522 12019 ts_manager.cc:194] Registered new tserver with Master: e24ec9dd61fa4f3494c0dd473742c4b9 (127.8.229.67:37733)
I20250905 07:49:03.222836 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006810357s
I20250905 07:49:03.223542 12019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48772
I20250905 07:49:04.128809 12146 heartbeater.cc:499] Master 127.8.229.126:40237 was elected leader, sending a full tablet report...
I20250905 07:49:04.190670 12221 heartbeater.cc:499] Master 127.8.229.126:40237 was elected leader, sending a full tablet report...
I20250905 07:49:04.224789 12295 heartbeater.cc:499] Master 127.8.229.126:40237 was elected leader, sending a full tablet report...
I20250905 07:49:04.257229 9109 test_util.cc:276] Using random seed: 308365510
I20250905 07:49:04.267786 12019 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48802:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:49:04.268424 12019 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:49:04.285383 12186 tablet_service.cc:1468] Processing CreateTablet for tablet b58e7ae6feff4c6a81cea8ee1d349fac (DEFAULT_TABLE table=test-workload [id=88b99908b7d64fdc84196790c74a981d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:49:04.285944 12186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58e7ae6feff4c6a81cea8ee1d349fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:04.286196 12111 tablet_service.cc:1468] Processing CreateTablet for tablet b58e7ae6feff4c6a81cea8ee1d349fac (DEFAULT_TABLE table=test-workload [id=88b99908b7d64fdc84196790c74a981d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:49:04.286531 12111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58e7ae6feff4c6a81cea8ee1d349fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:04.287388 12260 tablet_service.cc:1468] Processing CreateTablet for tablet b58e7ae6feff4c6a81cea8ee1d349fac (DEFAULT_TABLE table=test-workload [id=88b99908b7d64fdc84196790c74a981d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:49:04.287797 12260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58e7ae6feff4c6a81cea8ee1d349fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:04.291167 12316 tablet_bootstrap.cc:492] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de: Bootstrap starting.
I20250905 07:49:04.292533 12317 tablet_bootstrap.cc:492] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401: Bootstrap starting.
I20250905 07:49:04.292969 12316 tablet_bootstrap.cc:654] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:04.293880 12318 tablet_bootstrap.cc:492] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9: Bootstrap starting.
I20250905 07:49:04.294387 12317 tablet_bootstrap.cc:654] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:04.295368 12316 tablet_bootstrap.cc:492] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de: No bootstrap required, opened a new log
I20250905 07:49:04.295426 12318 tablet_bootstrap.cc:654] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:04.295749 12316 ts_tablet_manager.cc:1397] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20250905 07:49:04.296829 12316 raft_consensus.cc:357] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.297092 12316 raft_consensus.cc:383] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:04.297250 12316 raft_consensus.cc:738] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa000456317c43d88988041c854b91de, State: Initialized, Role: FOLLOWER
I20250905 07:49:04.297305 12318 tablet_bootstrap.cc:492] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9: No bootstrap required, opened a new log
I20250905 07:49:04.297503 12318 ts_tablet_manager.cc:1397] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:49:04.297627 12316 consensus_queue.cc:260] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.298126 12318 raft_consensus.cc:357] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.298375 12318 raft_consensus.cc:383] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:04.298506 12318 raft_consensus.cc:738] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e24ec9dd61fa4f3494c0dd473742c4b9, State: Initialized, Role: FOLLOWER
I20250905 07:49:04.298557 12316 ts_tablet_manager.cc:1428] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20250905 07:49:04.298820 12318 consensus_queue.cc:260] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.299533 12318 ts_tablet_manager.cc:1428] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:04.302695 12317 tablet_bootstrap.cc:492] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401: No bootstrap required, opened a new log
I20250905 07:49:04.302847 12317 ts_tablet_manager.cc:1397] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.005s
I20250905 07:49:04.303752 12317 raft_consensus.cc:357] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.304028 12317 raft_consensus.cc:383] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:04.304139 12317 raft_consensus.cc:738] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763ae09a04684fb2a6f15af0d5e38401, State: Initialized, Role: FOLLOWER
I20250905 07:49:04.304373 12317 consensus_queue.cc:260] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.304901 12317 ts_tablet_manager.cc:1428] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:04.362982 12323 raft_consensus.cc:491] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:04.363195 12323 raft_consensus.cc:513] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.364025 12323 leader_election.cc:290] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa000456317c43d88988041c854b91de (127.8.229.65:33521), 763ae09a04684fb2a6f15af0d5e38401 (127.8.229.66:36761)
I20250905 07:49:04.370286 12196 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b58e7ae6feff4c6a81cea8ee1d349fac" candidate_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "763ae09a04684fb2a6f15af0d5e38401" is_pre_election: true
I20250905 07:49:04.370569 12196 raft_consensus.cc:2466] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e24ec9dd61fa4f3494c0dd473742c4b9 in term 0.
I20250905 07:49:04.370582 12121 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b58e7ae6feff4c6a81cea8ee1d349fac" candidate_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa000456317c43d88988041c854b91de" is_pre_election: true
I20250905 07:49:04.370853 12121 raft_consensus.cc:2466] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e24ec9dd61fa4f3494c0dd473742c4b9 in term 0.
I20250905 07:49:04.371023 12236 leader_election.cc:304] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 763ae09a04684fb2a6f15af0d5e38401, e24ec9dd61fa4f3494c0dd473742c4b9; no voters:
I20250905 07:49:04.371264 12323 raft_consensus.cc:2802] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:04.371373 12323 raft_consensus.cc:491] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:04.371487 12323 raft_consensus.cc:3058] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:04.373333 12323 raft_consensus.cc:513] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.373848 12323 leader_election.cc:290] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [CANDIDATE]: Term 1 election: Requested vote from peers aa000456317c43d88988041c854b91de (127.8.229.65:33521), 763ae09a04684fb2a6f15af0d5e38401 (127.8.229.66:36761)
I20250905 07:49:04.374318 12196 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b58e7ae6feff4c6a81cea8ee1d349fac" candidate_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "763ae09a04684fb2a6f15af0d5e38401"
I20250905 07:49:04.374318 12121 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b58e7ae6feff4c6a81cea8ee1d349fac" candidate_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa000456317c43d88988041c854b91de"
I20250905 07:49:04.374528 12196 raft_consensus.cc:3058] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:04.374531 12121 raft_consensus.cc:3058] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:04.377282 12196 raft_consensus.cc:2466] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e24ec9dd61fa4f3494c0dd473742c4b9 in term 1.
I20250905 07:49:04.377282 12121 raft_consensus.cc:2466] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e24ec9dd61fa4f3494c0dd473742c4b9 in term 1.
I20250905 07:49:04.377740 12233 leader_election.cc:304] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa000456317c43d88988041c854b91de, e24ec9dd61fa4f3494c0dd473742c4b9; no voters:
I20250905 07:49:04.377982 12323 raft_consensus.cc:2802] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:04.378391 12323 raft_consensus.cc:695] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 1 LEADER]: Becoming Leader. State: Replica: e24ec9dd61fa4f3494c0dd473742c4b9, State: Running, Role: LEADER
I20250905 07:49:04.378685 12323 consensus_queue.cc:237] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } }
I20250905 07:49:04.381259 12019 catalog_manager.cc:5582] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e24ec9dd61fa4f3494c0dd473742c4b9 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e24ec9dd61fa4f3494c0dd473742c4b9" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37733 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:04.856076 12323 consensus_queue.cc:1035] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa000456317c43d88988041c854b91de" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:04.860962 12333 consensus_queue.cc:1035] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "763ae09a04684fb2a6f15af0d5e38401" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:05.009653 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:05.009835 12072 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:05.010412 12072 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:05.010557 12072 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:06.013052 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:07.011125 12072 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:07.011601 12072 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:07.011699 12072 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:07.016448 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:08.018733 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:09.012105 12072 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:09.012560 12072 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:09.012660 12072 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:09.021003 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:10.023337 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:11.013103 12072 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:11.013597 12072 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:11.013733 12072 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:11.025818 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:12.028307 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:13.014192 12072 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:13.014668 12072 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:13.014777 12072 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:13.030784 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:14.033339 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:15.015226 12072 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:15.015734 12072 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:15.015861 12072 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:15.035887 12067 placement_policy_util.cc:407] tablet b58e7ae6feff4c6a81cea8ee1d349fac: detected majority of replicas (2 of 3) at location L0
I20250905 07:49:15.473474 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:49:15.486487 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:15.486763 9109 tablet_replica.cc:331] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de: stopping tablet replica
I20250905 07:49:15.486946 9109 raft_consensus.cc:2241] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:15.487161 9109 raft_consensus.cc:2270] T b58e7ae6feff4c6a81cea8ee1d349fac P aa000456317c43d88988041c854b91de [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:15.502286 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:49:15.505120 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:49:15.514103 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:15.514343 9109 tablet_replica.cc:331] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401: stopping tablet replica
I20250905 07:49:15.514523 9109 raft_consensus.cc:2241] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:15.514712 9109 raft_consensus.cc:2270] T b58e7ae6feff4c6a81cea8ee1d349fac P 763ae09a04684fb2a6f15af0d5e38401 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250905 07:49:15.528829 12236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36761: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20250905 07:49:15.529913 12236 consensus_peers.cc:489] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 -> Peer 763ae09a04684fb2a6f15af0d5e38401 (127.8.229.66:36761): Couldn't send request to peer 763ae09a04684fb2a6f15af0d5e38401. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:15.535092 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:49:15.537863 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:49:15.546995 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:15.547219 9109 tablet_replica.cc:331] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9: stopping tablet replica
I20250905 07:49:15.547417 9109 raft_consensus.cc:2241] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:49:15.547744 9109 raft_consensus.cc:2270] T b58e7ae6feff4c6a81cea8ee1d349fac P e24ec9dd61fa4f3494c0dd473742c4b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:15.561306 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:49:15.564055 9109 master.cc:561] Master@127.8.229.126:40237 shutting down...
I20250905 07:49:15.572384 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:49:15.572624 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:15.572744 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 0904f9d5b8744c33b0c77ce212c1aaf5: stopping tablet replica
I20250905 07:49:15.586009 9109 master.cc:583] Master@127.8.229.126:40237 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy (12635 ms)
[ RUN ] AutoRebalancerTest.TestMaxMovesPerServer
I20250905 07:49:15.596861 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:45145
I20250905 07:49:15.597347 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:15.599969 12340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.600003 12341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.600178 12343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:15.600819 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:15.601066 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:15.601115 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:15.601154 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058555601151 us; error 0 us; skew 500 ppm
I20250905 07:49:15.601261 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:15.602325 9109 webserver.cc:480] Webserver started at http://127.8.229.126:42753/ using document root <none> and password file <none>
I20250905 07:49:15.602507 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:15.602558 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:15.602643 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:15.603201 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "0080ac993145405cb9ce0d80deff9150"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.605561 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:15.606937 12348 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.607308 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:15.607388 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "0080ac993145405cb9ce0d80deff9150"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.607517 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:15.624776 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:15.625280 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:15.635324 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:45145
I20250905 07:49:15.635344 12399 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:45145 every 8 connection(s)
I20250905 07:49:15.636680 12400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:15.640381 12400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150: Bootstrap starting.
I20250905 07:49:15.641613 12400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:15.643096 12400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150: No bootstrap required, opened a new log
I20250905 07:49:15.643688 12400 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0080ac993145405cb9ce0d80deff9150" member_type: VOTER }
I20250905 07:49:15.643810 12400 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:15.643870 12400 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080ac993145405cb9ce0d80deff9150, State: Initialized, Role: FOLLOWER
I20250905 07:49:15.644070 12400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0080ac993145405cb9ce0d80deff9150" member_type: VOTER }
I20250905 07:49:15.644178 12400 raft_consensus.cc:397] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:49:15.644245 12400 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:49:15.644331 12400 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:15.645912 12400 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0080ac993145405cb9ce0d80deff9150" member_type: VOTER }
I20250905 07:49:15.646088 12400 leader_election.cc:304] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0080ac993145405cb9ce0d80deff9150; no voters:
I20250905 07:49:15.646473 12400 leader_election.cc:290] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:49:15.646561 12403 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:15.646967 12403 raft_consensus.cc:695] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 1 LEADER]: Becoming Leader. State: Replica: 0080ac993145405cb9ce0d80deff9150, State: Running, Role: LEADER
I20250905 07:49:15.647183 12403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0080ac993145405cb9ce0d80deff9150" member_type: VOTER }
I20250905 07:49:15.647241 12400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:49:15.647847 12405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0080ac993145405cb9ce0d80deff9150. Latest consensus state: current_term: 1 leader_uuid: "0080ac993145405cb9ce0d80deff9150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0080ac993145405cb9ce0d80deff9150" member_type: VOTER } }
I20250905 07:49:15.647826 12404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0080ac993145405cb9ce0d80deff9150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0080ac993145405cb9ce0d80deff9150" member_type: VOTER } }
I20250905 07:49:15.647989 12405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [sys.catalog]: This master's current role is: LEADER
I20250905 07:49:15.648008 12404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [sys.catalog]: This master's current role is: LEADER
I20250905 07:49:15.649170 12407 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:49:15.650574 12407 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:49:15.652406 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:49:15.653277 12407 catalog_manager.cc:1349] Generated new cluster ID: b38915366d644605a6fa7417d9d68f8e
I20250905 07:49:15.653350 12407 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:49:15.666517 12407 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:49:15.667239 12407 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:49:15.676553 12407 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150: Generated new TSK 0
I20250905 07:49:15.676736 12407 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:49:15.685230 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:15.687676 12421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.687837 12422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.688280 12424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:15.688632 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:15.688889 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:15.688941 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:15.688962 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058555688959 us; error 0 us; skew 500 ppm
I20250905 07:49:15.689064 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:15.690150 9109 webserver.cc:480] Webserver started at http://127.8.229.65:45313/ using document root <none> and password file <none>
I20250905 07:49:15.690323 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:15.690377 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:15.690474 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:15.691046 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "2df0c6def6514c2a859a74187f5d2d8e"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.693475 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:49:15.694847 12429 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.695196 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:15.695272 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "2df0c6def6514c2a859a74187f5d2d8e"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.695379 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:15.711994 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:15.712478 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:15.712996 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:15.713788 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:15.713840 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.713909 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:15.713956 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.726244 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:35247
I20250905 07:49:15.726265 12491 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:35247 every 8 connection(s)
I20250905 07:49:15.727763 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:15.730628 12496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.731899 12497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.732442 12499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:15.732951 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:15.733227 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:15.733280 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:15.733317 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058555733313 us; error 0 us; skew 500 ppm
I20250905 07:49:15.733530 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:15.734730 9109 webserver.cc:480] Webserver started at http://127.8.229.66:33047/ using document root <none> and password file <none>
I20250905 07:49:15.734912 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:15.734968 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:15.735051 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:15.735628 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.737951 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20250905 07:49:15.738858 12492 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45145
I20250905 07:49:15.738981 12492 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:15.739387 12492 heartbeater.cc:507] Master 127.8.229.126:45145 requested a full tablet report, sending...
I20250905 07:49:15.739558 12504 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.740020 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:15.740111 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.740221 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:15.740386 12365 ts_manager.cc:194] Registered new tserver with Master: 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:15.741478 12365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37052
I20250905 07:49:15.748512 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:15.748965 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:15.749456 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:15.750156 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:15.750209 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.750291 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:15.750336 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.763371 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:36723
I20250905 07:49:15.763398 12566 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:36723 every 8 connection(s)
I20250905 07:49:15.764846 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:15.767802 12570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.768173 12571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:15.768339 12573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:15.769258 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:15.769546 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:15.769655 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:15.769742 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058555769737 us; error 0 us; skew 500 ppm
I20250905 07:49:15.769922 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:15.771523 12567 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45145
I20250905 07:49:15.771636 12567 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:15.771950 12567 heartbeater.cc:507] Master 127.8.229.126:45145 requested a full tablet report, sending...
I20250905 07:49:15.772708 12365 ts_manager.cc:194] Registered new tserver with Master: 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:15.773406 12365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37066
I20250905 07:49:15.775661 9109 webserver.cc:480] Webserver started at http://127.8.229.67:33369/ using document root <none> and password file <none>
I20250905 07:49:15.775856 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:15.775923 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:15.776039 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:15.776600 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "5531c3eeac82412fb18050ea7148b03e"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.778791 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20250905 07:49:15.780205 12578 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.780521 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:15.780611 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "5531c3eeac82412fb18050ea7148b03e"
format_stamp: "Formatted at 2025-09-05 07:49:15 on dist-test-slave-pgzt"
I20250905 07:49:15.780726 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:15.788422 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:15.788921 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:15.789449 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:15.790264 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:15.790321 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.790395 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:15.790433 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:15.803061 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:34511
I20250905 07:49:15.803095 12640 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:34511 every 8 connection(s)
I20250905 07:49:15.809144 12641 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45145
I20250905 07:49:15.809268 12641 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:15.809592 12641 heartbeater.cc:507] Master 127.8.229.126:45145 requested a full tablet report, sending...
I20250905 07:49:15.810305 12365 ts_manager.cc:194] Registered new tserver with Master: 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:15.810734 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006761362s
I20250905 07:49:15.811054 12365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37070
I20250905 07:49:16.742645 12492 heartbeater.cc:499] Master 127.8.229.126:45145 was elected leader, sending a full tablet report...
I20250905 07:49:16.774595 12567 heartbeater.cc:499] Master 127.8.229.126:45145 was elected leader, sending a full tablet report...
I20250905 07:49:16.812338 12641 heartbeater.cc:499] Master 127.8.229.126:45145 was elected leader, sending a full tablet report...
I20250905 07:49:16.838486 9109 test_util.cc:276] Using random seed: 320946766
I20250905 07:49:16.849606 12365 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37074:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu""\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:49:16.850322 12365 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:49:16.871605 12606 tablet_service.cc:1468] Processing CreateTablet for tablet f919efd17c674fe799365a7d79d969d1 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION VALUES < 178956970
I20250905 07:49:16.871785 12605 tablet_service.cc:1468] Processing CreateTablet for tablet 559347a6dcac472c9b09b80a16527175 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20250905 07:49:16.872145 12606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919efd17c674fe799365a7d79d969d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.872457 12604 tablet_service.cc:1468] Processing CreateTablet for tablet 3f902eed21ca4a1abaa2335e791c9fd5 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20250905 07:49:16.872820 12604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f902eed21ca4a1abaa2335e791c9fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.876447 12605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559347a6dcac472c9b09b80a16527175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.877750 12532 tablet_service.cc:1468] Processing CreateTablet for tablet f919efd17c674fe799365a7d79d969d1 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION VALUES < 178956970
I20250905 07:49:16.877848 12531 tablet_service.cc:1468] Processing CreateTablet for tablet 559347a6dcac472c9b09b80a16527175 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20250905 07:49:16.878168 12532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919efd17c674fe799365a7d79d969d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.878490 12530 tablet_service.cc:1468] Processing CreateTablet for tablet 3f902eed21ca4a1abaa2335e791c9fd5 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20250905 07:49:16.878878 12530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f902eed21ca4a1abaa2335e791c9fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.880625 12454 tablet_service.cc:1468] Processing CreateTablet for tablet c69a843a1d994abea9c6ec7321f6c3d7 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20250905 07:49:16.880826 12457 tablet_service.cc:1468] Processing CreateTablet for tablet f919efd17c674fe799365a7d79d969d1 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION VALUES < 178956970
I20250905 07:49:16.881053 12454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a843a1d994abea9c6ec7321f6c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.881248 12457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919efd17c674fe799365a7d79d969d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.881810 12526 tablet_service.cc:1468] Processing CreateTablet for tablet 0e94f73c59d74a2a862c445f8f70e618 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20250905 07:49:16.882145 12526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94f73c59d74a2a862c445f8f70e618. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.883780 12456 tablet_service.cc:1468] Processing CreateTablet for tablet 559347a6dcac472c9b09b80a16527175 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20250905 07:49:16.884177 12456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559347a6dcac472c9b09b80a16527175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.884117 12603 tablet_service.cc:1468] Processing CreateTablet for tablet c69a843a1d994abea9c6ec7321f6c3d7 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20250905 07:49:16.884541 12603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a843a1d994abea9c6ec7321f6c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.885020 12523 tablet_service.cc:1468] Processing CreateTablet for tablet 5209f5aac7a24f62a3b520b55ae51a28 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20250905 07:49:16.885417 12523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5209f5aac7a24f62a3b520b55ae51a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.887737 12602 tablet_service.cc:1468] Processing CreateTablet for tablet 89d6356d89d8404e94394907a2222a62 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20250905 07:49:16.887784 12529 tablet_service.cc:1468] Processing CreateTablet for tablet c69a843a1d994abea9c6ec7321f6c3d7 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20250905 07:49:16.888173 12602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d6356d89d8404e94394907a2222a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.888279 12529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a843a1d994abea9c6ec7321f6c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.888533 12452 tablet_service.cc:1468] Processing CreateTablet for tablet b380bffe76bd498e8d98e64dae8728ab (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20250905 07:49:16.888938 12452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b380bffe76bd498e8d98e64dae8728ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.888149 12453 tablet_service.cc:1468] Processing CreateTablet for tablet 89d6356d89d8404e94394907a2222a62 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20250905 07:49:16.891763 12528 tablet_service.cc:1468] Processing CreateTablet for tablet 89d6356d89d8404e94394907a2222a62 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20250905 07:49:16.892104 12528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d6356d89d8404e94394907a2222a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.892912 12450 tablet_service.cc:1468] Processing CreateTablet for tablet 39d264067d4447c592529d8bf6e39eac (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20250905 07:49:16.893232 12451 tablet_service.cc:1468] Processing CreateTablet for tablet 0e94f73c59d74a2a862c445f8f70e618 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20250905 07:49:16.893998 12449 tablet_service.cc:1468] Processing CreateTablet for tablet 655c91ccdcf4422a8795cecf130a56ed (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20250905 07:49:16.894733 12448 tablet_service.cc:1468] Processing CreateTablet for tablet 5209f5aac7a24f62a3b520b55ae51a28 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20250905 07:49:16.895269 12527 tablet_service.cc:1468] Processing CreateTablet for tablet b380bffe76bd498e8d98e64dae8728ab (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20250905 07:49:16.895819 12525 tablet_service.cc:1468] Processing CreateTablet for tablet 39d264067d4447c592529d8bf6e39eac (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20250905 07:49:16.896286 12525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d264067d4447c592529d8bf6e39eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.897053 12601 tablet_service.cc:1468] Processing CreateTablet for tablet b380bffe76bd498e8d98e64dae8728ab (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20250905 07:49:16.897562 12601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b380bffe76bd498e8d98e64dae8728ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.892104 12453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d6356d89d8404e94394907a2222a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.900383 12599 tablet_service.cc:1468] Processing CreateTablet for tablet 39d264067d4447c592529d8bf6e39eac (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20250905 07:49:16.900887 12600 tablet_service.cc:1468] Processing CreateTablet for tablet 0e94f73c59d74a2a862c445f8f70e618 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20250905 07:49:16.901029 12598 tablet_service.cc:1468] Processing CreateTablet for tablet 655c91ccdcf4422a8795cecf130a56ed (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20250905 07:49:16.887784 12455 tablet_service.cc:1468] Processing CreateTablet for tablet 3f902eed21ca4a1abaa2335e791c9fd5 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20250905 07:49:16.905252 12531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559347a6dcac472c9b09b80a16527175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.908058 12450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d264067d4447c592529d8bf6e39eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.910152 12448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5209f5aac7a24f62a3b520b55ae51a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.911845 12599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d264067d4447c592529d8bf6e39eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.912101 12524 tablet_service.cc:1468] Processing CreateTablet for tablet 655c91ccdcf4422a8795cecf130a56ed (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20250905 07:49:16.913401 12524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 655c91ccdcf4422a8795cecf130a56ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.913709 12604 tablet_service.cc:1468] Processing CreateTablet for tablet 3467a871458e47d488238ece84ae086b (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20250905 07:49:16.916067 12455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f902eed21ca4a1abaa2335e791c9fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.916270 12597 tablet_service.cc:1468] Processing CreateTablet for tablet 5209f5aac7a24f62a3b520b55ae51a28 (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20250905 07:49:16.917827 12532 tablet_service.cc:1468] Processing CreateTablet for tablet 3467a871458e47d488238ece84ae086b (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20250905 07:49:16.918308 12598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 655c91ccdcf4422a8795cecf130a56ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.918285 12662 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:16.922047 12597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5209f5aac7a24f62a3b520b55ae51a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.924404 12526 tablet_service.cc:1468] Processing CreateTablet for tablet 3a62c7bcdb1a497898d2078a2f891cff (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20250905 07:49:16.927592 12527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b380bffe76bd498e8d98e64dae8728ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.930298 12451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94f73c59d74a2a862c445f8f70e618. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.930980 12606 tablet_service.cc:1468] Processing CreateTablet for tablet 3a62c7bcdb1a497898d2078a2f891cff (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20250905 07:49:16.937654 12449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 655c91ccdcf4422a8795cecf130a56ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.939214 12454 tablet_service.cc:1468] Processing CreateTablet for tablet 3467a871458e47d488238ece84ae086b (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20250905 07:49:16.944365 12526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a62c7bcdb1a497898d2078a2f891cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.946276 12604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3467a871458e47d488238ece84ae086b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.948084 12600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94f73c59d74a2a862c445f8f70e618. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.948864 12454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3467a871458e47d488238ece84ae086b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.949389 12532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3467a871458e47d488238ece84ae086b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.948570 12450 tablet_service.cc:1468] Processing CreateTablet for tablet 3a62c7bcdb1a497898d2078a2f891cff (DEFAULT_TABLE table=test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20250905 07:49:16.972631 12606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a62c7bcdb1a497898d2078a2f891cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.977056 12450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a62c7bcdb1a497898d2078a2f891cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:16.988195 12661 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:16.989449 12662 tablet_bootstrap.cc:654] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:16.990501 12661 tablet_bootstrap.cc:654] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.002287 12662 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.002554 12662 ts_tablet_manager.cc:1397] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.084s user 0.004s sys 0.005s
I20250905 07:49:17.003422 12662 raft_consensus.cc:357] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.003795 12662 raft_consensus.cc:383] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.003944 12662 raft_consensus.cc:738] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.004297 12662 consensus_queue.cc:260] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.004941 12663 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.007058 12663 tablet_bootstrap.cc:654] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.007658 12661 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.007814 12661 ts_tablet_manager.cc:1397] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.011s
I20250905 07:49:17.008666 12662 ts_tablet_manager.cc:1428] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20250905 07:49:17.008795 12661 raft_consensus.cc:357] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.008980 12661 raft_consensus.cc:383] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.009055 12662 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.009075 12661 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.009354 12661 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.010885 12662 tablet_bootstrap.cc:654] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.013458 12663 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.013638 12663 ts_tablet_manager.cc:1397] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20250905 07:49:17.014639 12663 raft_consensus.cc:357] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.014878 12663 raft_consensus.cc:383] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.014982 12663 raft_consensus.cc:738] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.015045 12662 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.015190 12662 ts_tablet_manager.cc:1397] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20250905 07:49:17.015640 12663 consensus_queue.cc:260] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.015933 12661 ts_tablet_manager.cc:1428] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.008s user 0.000s sys 0.008s
I20250905 07:49:17.016036 12662 raft_consensus.cc:357] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.016230 12662 raft_consensus.cc:383] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.016327 12661 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.016332 12662 raft_consensus.cc:738] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.016623 12662 consensus_queue.cc:260] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.017102 12662 ts_tablet_manager.cc:1428] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.017407 12662 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.018313 12661 tablet_bootstrap.cc:654] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.020182 12662 tablet_bootstrap.cc:654] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.022152 12662 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.022296 12662 ts_tablet_manager.cc:1397] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250905 07:49:17.022825 12662 raft_consensus.cc:357] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.023209 12662 raft_consensus.cc:383] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.023319 12662 raft_consensus.cc:738] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.023607 12663 ts_tablet_manager.cc:1428] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.010s user 0.000s sys 0.007s
I20250905 07:49:17.023578 12662 consensus_queue.cc:260] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.023988 12662 ts_tablet_manager.cc:1428] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.024312 12662 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.026068 12662 tablet_bootstrap.cc:654] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.026461 12661 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.026608 12663 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.026719 12661 ts_tablet_manager.cc:1397] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.010s
I20250905 07:49:17.028427 12663 tablet_bootstrap.cc:654] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.028775 12661 raft_consensus.cc:357] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.028967 12661 raft_consensus.cc:383] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.029058 12661 raft_consensus.cc:738] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.029331 12662 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.029364 12661 consensus_queue.cc:260] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.029520 12662 ts_tablet_manager.cc:1397] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20250905 07:49:17.029978 12661 ts_tablet_manager.cc:1428] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20250905 07:49:17.030256 12661 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.030210 12662 raft_consensus.cc:357] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.030406 12662 raft_consensus.cc:383] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.030421 12663 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.030515 12662 raft_consensus.cc:738] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.030712 12663 ts_tablet_manager.cc:1397] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:49:17.030803 12662 consensus_queue.cc:260] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.031338 12662 ts_tablet_manager.cc:1428] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:49:17.031658 12662 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.032055 12661 tablet_bootstrap.cc:654] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.033453 12662 tablet_bootstrap.cc:654] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.034380 12663 raft_consensus.cc:357] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.034636 12663 raft_consensus.cc:383] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.034790 12663 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.035030 12663 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.035187 12662 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.035326 12662 ts_tablet_manager.cc:1397] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:49:17.035698 12663 ts_tablet_manager.cc:1428] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250905 07:49:17.035928 12661 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.035955 12663 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.036067 12661 ts_tablet_manager.cc:1397] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250905 07:49:17.036027 12662 raft_consensus.cc:357] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.036195 12662 raft_consensus.cc:383] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.036278 12662 raft_consensus.cc:738] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.036561 12662 consensus_queue.cc:260] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.036749 12661 raft_consensus.cc:357] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.036913 12661 raft_consensus.cc:383] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.036998 12661 raft_consensus.cc:738] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.037108 12662 ts_tablet_manager.cc:1428] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.037243 12661 consensus_queue.cc:260] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.037408 12662 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.037776 12661 ts_tablet_manager.cc:1428] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:49:17.038486 12663 tablet_bootstrap.cc:654] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.040421 12662 tablet_bootstrap.cc:654] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.045408 12663 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.045547 12663 ts_tablet_manager.cc:1397] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20250905 07:49:17.045663 12662 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.045787 12662 ts_tablet_manager.cc:1397] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20250905 07:49:17.046288 12662 raft_consensus.cc:357] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.046247 12663 raft_consensus.cc:357] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.046445 12662 raft_consensus.cc:383] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.046480 12663 raft_consensus.cc:383] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.046504 12662 raft_consensus.cc:738] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.046530 12663 raft_consensus.cc:738] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.046707 12662 consensus_queue.cc:260] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.046725 12663 consensus_queue.cc:260] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.047219 12663 ts_tablet_manager.cc:1428] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:49:17.047248 12662 ts_tablet_manager.cc:1428] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:17.047497 12663 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.047538 12662 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.048702 12661 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.049008 12663 tablet_bootstrap.cc:654] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.049013 12662 tablet_bootstrap.cc:654] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.049924 12661 tablet_bootstrap.cc:654] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.053764 12662 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.053884 12662 ts_tablet_manager.cc:1397] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20250905 07:49:17.054525 12662 raft_consensus.cc:357] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.054689 12662 raft_consensus.cc:383] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.054741 12662 raft_consensus.cc:738] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.054955 12662 consensus_queue.cc:260] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.055903 12662 ts_tablet_manager.cc:1428] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:49:17.056149 12662 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.057775 12662 tablet_bootstrap.cc:654] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.058504 12666 raft_consensus.cc:491] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.058616 12666 raft_consensus.cc:513] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.059504 12666 leader_election.cc:290] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511), 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:17.060994 12663 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.061115 12663 ts_tablet_manager.cc:1397] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20250905 07:49:17.061300 12661 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.061436 12661 ts_tablet_manager.cc:1397] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20250905 07:49:17.061744 12663 raft_consensus.cc:357] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.061895 12663 raft_consensus.cc:383] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.061971 12663 raft_consensus.cc:738] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.061965 12661 raft_consensus.cc:357] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.062094 12661 raft_consensus.cc:383] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.062156 12661 raft_consensus.cc:738] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.062206 12663 consensus_queue.cc:260] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.062299 12661 consensus_queue.cc:260] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.062620 12663 ts_tablet_manager.cc:1428] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250905 07:49:17.062884 12663 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.063208 12661 ts_tablet_manager.cc:1428] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:49:17.063422 12661 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.064149 12663 tablet_bootstrap.cc:654] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.064554 12661 tablet_bootstrap.cc:654] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.071964 12663 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.072078 12663 ts_tablet_manager.cc:1397] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20250905 07:49:17.072252 12661 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.072432 12661 ts_tablet_manager.cc:1397] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20250905 07:49:17.072660 12663 raft_consensus.cc:357] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.072801 12663 raft_consensus.cc:383] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.072856 12663 raft_consensus.cc:738] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.073030 12663 consensus_queue.cc:260] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.073390 12663 ts_tablet_manager.cc:1428] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:49:17.073575 12663 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.074555 12663 tablet_bootstrap.cc:654] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.075661 12661 raft_consensus.cc:357] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.075855 12661 raft_consensus.cc:383] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.075945 12661 raft_consensus.cc:738] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.076198 12661 consensus_queue.cc:260] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.076668 12661 ts_tablet_manager.cc:1428] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250905 07:49:17.077627 12661 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.077977 12666 raft_consensus.cc:491] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.078096 12666 raft_consensus.cc:513] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.078639 12666 leader_election.cc:290] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.079061 12661 tablet_bootstrap.cc:654] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.080111 12668 raft_consensus.cc:491] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.080265 12668 raft_consensus.cc:513] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.081270 12668 leader_election.cc:290] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.089429 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.089707 12616 raft_consensus.cc:2466] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.090090 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.090320 12615 raft_consensus.cc:2466] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.090857 12508 leader_election.cc:304] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.091197 12508 leader_election.cc:304] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.091632 12666 raft_consensus.cc:2802] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.091725 12666 raft_consensus.cc:491] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.091816 12666 raft_consensus.cc:3058] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.095281 12663 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.095288 12666 raft_consensus.cc:513] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.095503 12663 ts_tablet_manager.cc:1397] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s
I20250905 07:49:17.096159 12666 leader_election.cc:290] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Requested vote from peers 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511), 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:17.096366 12666 raft_consensus.cc:2802] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.096441 12663 raft_consensus.cc:357] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.096455 12666 raft_consensus.cc:491] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.096686 12663 raft_consensus.cc:383] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.096694 12666 raft_consensus.cc:3058] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.096760 12663 raft_consensus.cc:738] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.097126 12663 consensus_queue.cc:260] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.099982 12663 ts_tablet_manager.cc:1428] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250905 07:49:17.100194 12666 raft_consensus.cc:513] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.101482 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:17.101671 12615 raft_consensus.cc:3058] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.102326 12663 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.104035 12663 tablet_bootstrap.cc:654] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.105978 12615 raft_consensus.cc:2466] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 1.
I20250905 07:49:17.106525 12508 leader_election.cc:304] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.108984 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.109278 12542 raft_consensus.cc:2466] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 0.
I20250905 07:49:17.109395 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:17.109591 12615 raft_consensus.cc:3058] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.109808 12581 leader_election.cc:304] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.110121 12668 raft_consensus.cc:2802] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.110242 12668 raft_consensus.cc:491] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.110339 12668 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.113139 12615 raft_consensus.cc:2466] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 1.
I20250905 07:49:17.113613 12508 leader_election.cc:304] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.113852 12668 raft_consensus.cc:513] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.115213 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.115417 12542 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.115746 12692 raft_consensus.cc:2802] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.117733 12467 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.118019 12467 raft_consensus.cc:2466] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.118619 12467 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.119038 12542 raft_consensus.cc:2466] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 1.
I20250905 07:49:17.119172 12467 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.119354 12467 raft_consensus.cc:3058] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.119480 12581 leader_election.cc:304] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.120312 12668 leader_election.cc:290] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.120494 12668 raft_consensus.cc:2802] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.120609 12668 raft_consensus.cc:695] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:17.120920 12668 consensus_queue.cc:237] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.122910 12467 raft_consensus.cc:2466] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 1.
W20250905 07:49:17.123865 12506 leader_election.cc:343] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:17.124680 12467 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
W20250905 07:49:17.125111 12506 leader_election.cc:343] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:17.125393 12666 leader_election.cc:290] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.125577 12666 raft_consensus.cc:2802] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.125694 12666 raft_consensus.cc:695] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:17.125980 12666 consensus_queue.cc:237] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.132627 12692 raft_consensus.cc:695] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:17.132997 12692 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } }
I20250905 07:49:17.136513 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.136827 12466 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.136500 12467 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.141396 12466 raft_consensus.cc:2466] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 1.
I20250905 07:49:17.142192 12670 raft_consensus.cc:491] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.142325 12670 raft_consensus.cc:513] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.142539 12662 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.142683 12662 ts_tablet_manager.cc:1397] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.002s
I20250905 07:49:17.143294 12670 leader_election.cc:290] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.143384 12662 raft_consensus.cc:357] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.143563 12662 raft_consensus.cc:383] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.143632 12662 raft_consensus.cc:738] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.143865 12662 consensus_queue.cc:260] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.144393 12662 ts_tablet_manager.cc:1428] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.144688 12662 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.146486 12662 tablet_bootstrap.cc:654] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.154572 12696 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.154767 12696 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.155786 12696 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.157977 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
W20250905 07:49:17.160008 12581 leader_election.cc:343] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:17.160665 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.160866 12466 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 0.
I20250905 07:49:17.161753 12580 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters: 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:17.162245 12698 raft_consensus.cc:2802] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.162350 12698 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.162436 12698 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.166067 12698 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.167135 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.167332 12466 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.169874 12694 raft_consensus.cc:491] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.170037 12694 raft_consensus.cc:513] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.170538 12466 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 1.
I20250905 07:49:17.171350 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.171584 12466 raft_consensus.cc:2466] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.171662 12580 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.171761 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.171905 12696 raft_consensus.cc:2802] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.172247 12506 leader_election.cc:304] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:17.172498 12666 raft_consensus.cc:2802] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.172573 12666 raft_consensus.cc:491] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.172648 12666 raft_consensus.cc:3058] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.172685 12661 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.172880 12661 ts_tablet_manager.cc:1397] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.095s user 0.032s sys 0.059s
I20250905 07:49:17.173662 12661 raft_consensus.cc:357] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.173866 12661 raft_consensus.cc:383] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.173971 12661 raft_consensus.cc:738] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.174223 12661 consensus_queue.cc:260] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.174126 12364 catalog_manager.cc:5582] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:17.174821 12661 ts_tablet_manager.cc:1428] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:49:17.175137 12661 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.175207 12666 raft_consensus.cc:513] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.175689 12666 leader_election.cc:290] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.175762 12698 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.175697 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.176450 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.176594 12466 raft_consensus.cc:3058] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
W20250905 07:49:17.176697 12581 leader_election.cc:343] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:17.177721 12661 tablet_bootstrap.cc:654] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.180090 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
W20250905 07:49:17.180693 12508 leader_election.cc:343] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): Illegal state: must be running to vote when last-logged opid is not known
W20250905 07:49:17.181017 12508 leader_election.cc:343] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:17.181455 12466 raft_consensus.cc:2466] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 1.
I20250905 07:49:17.181958 12506 leader_election.cc:304] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters: 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:17.182515 12666 raft_consensus.cc:2802] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.182786 12666 raft_consensus.cc:695] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:17.183143 12666 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.184597 12694 leader_election.cc:290] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.175215 12662 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.185667 12696 raft_consensus.cc:695] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:17.185704 12662 ts_tablet_manager.cc:1397] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.000s
I20250905 07:49:17.185953 12696 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.186607 12662 raft_consensus.cc:357] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.186841 12662 raft_consensus.cc:383] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.186945 12662 raft_consensus.cc:738] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.187029 12364 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 0 to 1, leader changed from <none> to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:17.187239 12662 consensus_queue.cc:260] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.187841 12662 ts_tablet_manager.cc:1428] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.187700 12364 catalog_manager.cc:5582] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 0 to 1, leader changed from <none> to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:17.188170 12662 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.188261 12661 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.188434 12661 ts_tablet_manager.cc:1397] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.001s
I20250905 07:49:17.189152 12661 raft_consensus.cc:357] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.189375 12661 raft_consensus.cc:383] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.189505 12661 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.189742 12662 tablet_bootstrap.cc:654] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.189791 12692 raft_consensus.cc:491] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.189865 12661 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.189949 12692 raft_consensus.cc:513] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.190683 12692 leader_election.cc:290] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.191239 12661 ts_tablet_manager.cc:1428] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20250905 07:49:17.191295 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.191643 12661 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.191670 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
W20250905 07:49:17.192107 12508 leader_election.cc:343] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): Illegal state: must be running to vote when last-logged opid is not known
W20250905 07:49:17.192457 12506 leader_election.cc:343] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:17.192586 12506 leader_election.cc:304] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a; no voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:17.192951 12692 raft_consensus.cc:2747] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:49:17.193133 12365 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:17.193574 12661 tablet_bootstrap.cc:654] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.194049 12363 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 0 to 1, leader changed from <none> to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:17.197104 12663 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.197265 12663 ts_tablet_manager.cc:1397] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.095s user 0.020s sys 0.021s
I20250905 07:49:17.199378 12663 raft_consensus.cc:357] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.199590 12663 raft_consensus.cc:383] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.199700 12663 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.199921 12663 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.200455 12663 ts_tablet_manager.cc:1428] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250905 07:49:17.200747 12663 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.202147 12663 tablet_bootstrap.cc:654] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.202185 12661 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.202489 12661 ts_tablet_manager.cc:1397] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20250905 07:49:17.203524 12661 raft_consensus.cc:357] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.203847 12661 raft_consensus.cc:383] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.203989 12661 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.204602 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.204838 12615 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 0.
I20250905 07:49:17.205052 12663 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.205199 12663 ts_tablet_manager.cc:1397] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250905 07:49:17.205524 12433 leader_election.cc:304] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.205911 12663 raft_consensus.cc:357] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.206089 12663 raft_consensus.cc:383] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.206115 12670 raft_consensus.cc:2802] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.206185 12663 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.206205 12670 raft_consensus.cc:491] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.206318 12670 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.205992 12661 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.206456 12663 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.207357 12663 ts_tablet_manager.cc:1428] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.207633 12662 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.207768 12663 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.207780 12662 ts_tablet_manager.cc:1397] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.009s
I20250905 07:49:17.208328 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.208525 12542 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 0.
I20250905 07:49:17.208531 12662 raft_consensus.cc:357] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.208750 12662 raft_consensus.cc:383] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.208859 12662 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.209138 12662 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.209404 12670 raft_consensus.cc:513] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.209781 12662 ts_tablet_manager.cc:1428] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:49:17.210108 12662 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.210212 12661 ts_tablet_manager.cc:1428] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20250905 07:49:17.210390 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.210582 12542 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.211813 12662 tablet_bootstrap.cc:654] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.212479 12670 leader_election.cc:290] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.209781 12663 tablet_bootstrap.cc:654] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.213526 12542 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 1.
I20250905 07:49:17.213968 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:17.214174 12615 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.214915 12662 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.215014 12662 ts_tablet_manager.cc:1397] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250905 07:49:17.215562 12662 raft_consensus.cc:357] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.215747 12662 raft_consensus.cc:383] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.215829 12662 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.216075 12662 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.217022 12432 leader_election.cc:304] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:17.210578 12661 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.217321 12711 raft_consensus.cc:2802] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.217505 12615 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 1.
I20250905 07:49:17.217916 12711 raft_consensus.cc:695] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:17.218271 12711 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.218676 12662 ts_tablet_manager.cc:1428] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250905 07:49:17.219265 12661 tablet_bootstrap.cc:654] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.221042 12662 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: Bootstrap starting.
I20250905 07:49:17.221218 12364 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 1 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:17.221534 12661 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.221680 12661 ts_tablet_manager.cc:1397] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20250905 07:49:17.222440 12661 raft_consensus.cc:357] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.222568 12663 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.222618 12661 raft_consensus.cc:383] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.222709 12661 raft_consensus.cc:738] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.222708 12663 ts_tablet_manager.cc:1397] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.001s
I20250905 07:49:17.222815 12662 tablet_bootstrap.cc:654] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.223003 12661 consensus_queue.cc:260] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.223552 12663 raft_consensus.cc:357] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.223712 12663 raft_consensus.cc:383] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.223815 12663 raft_consensus.cc:738] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.224051 12663 consensus_queue.cc:260] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.224543 12663 ts_tablet_manager.cc:1428] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:49:17.225226 12661 ts_tablet_manager.cc:1428] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:49:17.225521 12661 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.225836 12662 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: No bootstrap required, opened a new log
I20250905 07:49:17.226028 12662 ts_tablet_manager.cc:1397] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20250905 07:49:17.226876 12662 raft_consensus.cc:357] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.227123 12662 raft_consensus.cc:383] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.227279 12662 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.227460 12663 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.227177 12661 tablet_bootstrap.cc:654] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.228358 12662 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.228760 12662 ts_tablet_manager.cc:1428] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250905 07:49:17.229198 12663 tablet_bootstrap.cc:654] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.230847 12661 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.230994 12661 ts_tablet_manager.cc:1397] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20250905 07:49:17.231134 12663 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.231278 12663 ts_tablet_manager.cc:1397] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:49:17.231779 12661 raft_consensus.cc:357] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.231891 12663 raft_consensus.cc:357] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.231971 12661 raft_consensus.cc:383] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.232112 12661 raft_consensus.cc:738] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.232146 12663 raft_consensus.cc:383] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.232242 12663 raft_consensus.cc:738] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.232380 12661 consensus_queue.cc:260] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.232470 12663 consensus_queue.cc:260] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.232882 12661 ts_tablet_manager.cc:1428] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.233143 12661 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.233480 12663 ts_tablet_manager.cc:1428] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:49:17.233770 12663 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.234910 12661 tablet_bootstrap.cc:654] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.235514 12663 tablet_bootstrap.cc:654] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.236873 12661 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.237036 12661 ts_tablet_manager.cc:1397] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20250905 07:49:17.237803 12661 raft_consensus.cc:357] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.237967 12661 raft_consensus.cc:383] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.238062 12661 raft_consensus.cc:738] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.238307 12661 consensus_queue.cc:260] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.238519 12663 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.238655 12663 ts_tablet_manager.cc:1397] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20250905 07:49:17.238811 12661 ts_tablet_manager.cc:1428] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:49:17.239120 12661 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e: Bootstrap starting.
I20250905 07:49:17.239370 12663 raft_consensus.cc:357] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.239580 12663 raft_consensus.cc:383] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.239701 12663 raft_consensus.cc:738] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.239952 12663 consensus_queue.cc:260] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.240559 12663 ts_tablet_manager.cc:1428] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:17.240885 12663 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e: Bootstrap starting.
I20250905 07:49:17.240862 12661 tablet_bootstrap.cc:654] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.242904 12663 tablet_bootstrap.cc:654] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:17.244899 12663 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e: No bootstrap required, opened a new log
I20250905 07:49:17.245107 12663 ts_tablet_manager.cc:1397] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:49:17.245385 12661 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e: No bootstrap required, opened a new log
I20250905 07:49:17.245518 12661 ts_tablet_manager.cc:1397] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20250905 07:49:17.245712 12663 raft_consensus.cc:357] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.245879 12663 raft_consensus.cc:383] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.245961 12663 raft_consensus.cc:738] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.246134 12661 raft_consensus.cc:357] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.246201 12663 consensus_queue.cc:260] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.246291 12661 raft_consensus.cc:383] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:17.246404 12661 raft_consensus.cc:738] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Initialized, Role: FOLLOWER
I20250905 07:49:17.246681 12663 ts_tablet_manager.cc:1428] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:49:17.246678 12661 consensus_queue.cc:260] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.247208 12661 ts_tablet_manager.cc:1428] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:49:17.297968 12692 raft_consensus.cc:491] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.298238 12692 raft_consensus.cc:513] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.298882 12692 leader_election.cc:290] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.299409 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.299450 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.299676 12466 raft_consensus.cc:2466] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.299682 12615 raft_consensus.cc:2466] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.300077 12506 leader_election.cc:304] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:17.300340 12692 raft_consensus.cc:2802] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.300437 12692 raft_consensus.cc:491] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.300551 12692 raft_consensus.cc:3058] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.302481 12692 raft_consensus.cc:513] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.303062 12692 leader_election.cc:290] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.303426 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:17.303469 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.303651 12615 raft_consensus.cc:3058] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.303699 12466 raft_consensus.cc:3058] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.305529 12615 raft_consensus.cc:2466] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 1.
I20250905 07:49:17.305593 12466 raft_consensus.cc:2466] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 1.
I20250905 07:49:17.305931 12508 leader_election.cc:304] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.306191 12692 raft_consensus.cc:2802] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.306384 12692 raft_consensus.cc:695] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:17.306628 12692 consensus_queue.cc:237] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.309046 12365 catalog_manager.cc:5582] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 0 to 1, leader changed from <none> to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:17.325501 12711 raft_consensus.cc:491] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.325809 12670 raft_consensus.cc:491] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.325754 12711 raft_consensus.cc:513] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.325942 12670 raft_consensus.cc:513] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.326509 12670 leader_election.cc:290] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.326508 12711 leader_election.cc:290] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.327036 12615 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.327214 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.327250 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.327214 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.327340 12615 raft_consensus.cc:2391] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2df0c6def6514c2a859a74187f5d2d8e in current term 1: Already voted for candidate 069247fbe8b34bbd81d3549f0fc0b93a in this term.
I20250905 07:49:17.327486 12542 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 0.
I20250905 07:49:17.327576 12616 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 0.
I20250905 07:49:17.327895 12432 leader_election.cc:304] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:17.328176 12711 raft_consensus.cc:2802] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.328269 12711 raft_consensus.cc:491] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.328382 12711 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.328339 12432 leader_election.cc:304] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e; no voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:17.328609 12670 raft_consensus.cc:3058] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.331295 12670 raft_consensus.cc:2747] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:49:17.331211 12711 raft_consensus.cc:513] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.331842 12711 leader_election.cc:290] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.332213 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:17.332286 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.332350 12616 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.332517 12541 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.334340 12616 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 1.
I20250905 07:49:17.334528 12541 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 1.
I20250905 07:49:17.334690 12433 leader_election.cc:304] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.334954 12711 raft_consensus.cc:2802] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.335091 12711 raft_consensus.cc:695] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:17.335350 12711 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.337819 12365 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 1 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:17.408294 12704 raft_consensus.cc:491] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.408550 12704 raft_consensus.cc:513] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.409235 12704 leader_election.cc:290] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.409772 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.409794 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.409989 12541 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 0.
I20250905 07:49:17.409971 12466 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 0.
I20250905 07:49:17.410423 12581 leader_election.cc:304] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.410661 12704 raft_consensus.cc:2802] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.410768 12704 raft_consensus.cc:491] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.410858 12704 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.412760 12704 raft_consensus.cc:513] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.413291 12704 leader_election.cc:290] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.413731 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.413733 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.413939 12541 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.413949 12466 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.415786 12541 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 1.
I20250905 07:49:17.415815 12466 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 1.
I20250905 07:49:17.416160 12581 leader_election.cc:304] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.416448 12704 raft_consensus.cc:2802] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.416572 12704 raft_consensus.cc:695] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:17.416836 12704 consensus_queue.cc:237] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.419067 12365 catalog_manager.cc:5582] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:17.439697 12711 raft_consensus.cc:491] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.439934 12711 raft_consensus.cc:513] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.440627 12711 leader_election.cc:290] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.441118 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3467a871458e47d488238ece84ae086b" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.441126 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3467a871458e47d488238ece84ae086b" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.441357 12616 raft_consensus.cc:2466] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 0.
I20250905 07:49:17.441416 12541 raft_consensus.cc:2466] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 0.
I20250905 07:49:17.441804 12432 leader_election.cc:304] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:17.442086 12711 raft_consensus.cc:2802] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.442209 12711 raft_consensus.cc:491] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.442296 12711 raft_consensus.cc:3058] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.444209 12711 raft_consensus.cc:513] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.444725 12711 leader_election.cc:290] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.445127 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3467a871458e47d488238ece84ae086b" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.445142 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3467a871458e47d488238ece84ae086b" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:17.445310 12541 raft_consensus.cc:3058] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.445369 12616 raft_consensus.cc:3058] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.447098 12616 raft_consensus.cc:2466] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 1.
I20250905 07:49:17.447114 12541 raft_consensus.cc:2466] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 1.
I20250905 07:49:17.447496 12433 leader_election.cc:304] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.447808 12711 raft_consensus.cc:2802] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.447938 12711 raft_consensus.cc:695] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:17.448205 12711 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.450757 12365 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 1 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:17.469554 12692 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.469769 12692 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.470484 12692 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.471083 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.471038 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.471307 12466 raft_consensus.cc:2391] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 069247fbe8b34bbd81d3549f0fc0b93a in current term 1: Already voted for candidate 5531c3eeac82412fb18050ea7148b03e in this term.
I20250905 07:49:17.471684 12506 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a; no voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:17.471935 12692 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.473728 12692 raft_consensus.cc:2747] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:49:17.482409 12692 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.482628 12692 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.483333 12692 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.483686 12704 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.483937 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.483970 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e" is_pre_election: true
I20250905 07:49:17.483948 12704 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.484155 12466 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.484277 12616 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 0.
I20250905 07:49:17.484602 12506 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:17.484897 12692 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.484933 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.485019 12692 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.485077 12704 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.485146 12466 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 0.
I20250905 07:49:17.485152 12692 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.485496 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.485541 12580 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.485802 12704 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.485888 12704 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.485996 12704 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.487604 12692 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.487928 12541 raft_consensus.cc:2391] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5531c3eeac82412fb18050ea7148b03e in current term 1: Already voted for candidate 069247fbe8b34bbd81d3549f0fc0b93a in this term.
I20250905 07:49:17.487996 12704 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.488638 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.488705 12616 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:17.488790 12704 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.488834 12466 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.488952 12616 raft_consensus.cc:2391] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 069247fbe8b34bbd81d3549f0fc0b93a in current term 1: Already voted for candidate 5531c3eeac82412fb18050ea7148b03e in this term.
I20250905 07:49:17.489401 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.489473 12692 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:17.489450 12467 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.489653 12541 raft_consensus.cc:2391] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5531c3eeac82412fb18050ea7148b03e in current term 1: Already voted for candidate 069247fbe8b34bbd81d3549f0fc0b93a in this term.
I20250905 07:49:17.490003 12580 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5531c3eeac82412fb18050ea7148b03e; no voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:17.490278 12704 raft_consensus.cc:2747] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250905 07:49:17.491291 12466 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 1.
I20250905 07:49:17.491663 12506 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters: 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:17.491906 12692 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.492061 12692 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:17.492314 12692 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.494673 12365 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 0 to 1, leader changed from <none> to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:17.527132 12704 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.531594 12696 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.566195 12696 raft_consensus.cc:491] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:17.566437 12696 raft_consensus.cc:513] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.567006 12696 leader_election.cc:290] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.567540 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" is_pre_election: true
I20250905 07:49:17.567536 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e" is_pre_election: true
I20250905 07:49:17.567726 12541 raft_consensus.cc:2466] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 0.
I20250905 07:49:17.567793 12466 raft_consensus.cc:2466] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 0.
I20250905 07:49:17.568123 12581 leader_election.cc:304] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.568357 12696 raft_consensus.cc:2802] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:17.568444 12696 raft_consensus.cc:491] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:17.568545 12696 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.570353 12696 raft_consensus.cc:513] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.570930 12696 leader_election.cc:290] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:17.571290 12466 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:17.571405 12541 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:17.571537 12466 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.571614 12541 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.574239 12466 raft_consensus.cc:2466] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 1.
I20250905 07:49:17.574245 12541 raft_consensus.cc:2466] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 1.
I20250905 07:49:17.574625 12581 leader_election.cc:304] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:17.574872 12696 raft_consensus.cc:2802] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:17.575029 12696 raft_consensus.cc:695] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:17.575301 12696 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:17.577867 12364 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:17.647815 12616 raft_consensus.cc:3058] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:17.650386 12692 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.652347 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:17.659642 12692 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.662441 12696 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.664171 12670 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.678104 12692 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.680083 12467 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:17.681464 12704 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.682888 12541 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:17.684823 12704 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.685281 12704 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.696050 12539 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:17.742738 12704 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.760630 12463 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:17.765033 12696 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.808758 12692 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.809847 12743 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20250905 07:49:17.859665 12670 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20250905 07:49:17.866137 12692 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20250905 07:49:17.899550 12666 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20250905 07:49:17.939560 12743 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.949939 12743 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:17.956138 12542 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:17.968993 12542 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:17.988497 12694 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:18.025738 12692 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.049s
I20250905 07:49:18.035521 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:18.041985 12743 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20250905 07:49:18.046236 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:18.067082 12692 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:18.067303 12692 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:18.067780 12692 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:18.082821 12466 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59180
I20250905 07:49:18.090209 12666 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20250905 07:49:18.092051 12466 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:18.092235 12466 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
W20250905 07:49:18.121943 12432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.229.66:36723, user_credentials={real_user=slave}} blocked reactor thread for 72498us
I20250905 07:49:18.129886 12466 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:18.132953 12466 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:18.133190 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
W20250905 07:49:18.156680 12521 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Service unavailable: leader transfer in progress
W20250905 07:49:18.175554 12516 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Service unavailable: leader transfer in progress
W20250905 07:49:18.179826 12521 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Service unavailable: leader transfer in progress
I20250905 07:49:18.195055 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:18.195380 12542 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:18.195559 12542 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:18.203102 12542 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
W20250905 07:49:18.204013 12513 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
I20250905 07:49:18.206998 12542 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
W20250905 07:49:18.209550 12515 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
W20250905 07:49:18.215629 12513 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
W20250905 07:49:18.226172 12514 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
W20250905 07:49:18.233430 12440 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:59196: Illegal state: replica 2df0c6def6514c2a859a74187f5d2d8e is not leader of this config: current role FOLLOWER
W20250905 07:49:18.242182 12447 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:59196: Illegal state: replica 2df0c6def6514c2a859a74187f5d2d8e is not leader of this config: current role FOLLOWER
W20250905 07:49:18.254715 12591 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:45292: Illegal state: replica 5531c3eeac82412fb18050ea7148b03e is not leader of this config: current role FOLLOWER
W20250905 07:49:18.244262 12446 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:59196: Illegal state: replica 2df0c6def6514c2a859a74187f5d2d8e is not leader of this config: current role FOLLOWER
W20250905 07:49:18.259750 12447 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:59196: Illegal state: replica 2df0c6def6514c2a859a74187f5d2d8e is not leader of this config: current role FOLLOWER
W20250905 07:49:18.264004 12590 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:45292: Illegal state: replica 5531c3eeac82412fb18050ea7148b03e is not leader of this config: current role FOLLOWER
W20250905 07:49:18.284164 12591 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:45292: Illegal state: replica 5531c3eeac82412fb18050ea7148b03e is not leader of this config: current role FOLLOWER
W20250905 07:49:18.285429 12590 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:45292: Illegal state: replica 5531c3eeac82412fb18050ea7148b03e is not leader of this config: current role FOLLOWER
I20250905 07:49:18.286955 12542 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 2.
I20250905 07:49:18.292162 12432 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters: 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:18.292730 12789 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:18.292893 12789 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
W20250905 07:49:18.293251 12513 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
I20250905 07:49:18.293196 12789 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
W20250905 07:49:18.293345 12514 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
I20250905 07:49:18.299765 12359 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 1 to 2, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:18.303121 12615 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
W20250905 07:49:18.304803 12514 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
W20250905 07:49:18.305716 12514 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:58408: Illegal state: replica 069247fbe8b34bbd81d3549f0fc0b93a is not leader of this config: current role FOLLOWER
I20250905 07:49:18.306149 12615 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:49:18.306638 12791 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:49:18.308792 12542 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:49:18.309607 12787 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250905 07:49:18.826664 12432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.229.66:36723, user_credentials={real_user=slave}} blocked reactor thread for 75269.2us
I20250905 07:49:19.179980 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:19.188853 12813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:19.189221 12814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:19.191627 9109 server_base.cc:1047] running on GCE node
W20250905 07:49:19.192422 12816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:19.192721 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:19.192778 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:19.192802 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058559192799 us; error 0 us; skew 500 ppm
I20250905 07:49:19.192905 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:19.194053 9109 webserver.cc:480] Webserver started at http://127.8.229.68:37603/ using document root <none> and password file <none>
I20250905 07:49:19.194263 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:19.194315 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:19.194417 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:19.194983 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "0e02ae2f2eb74d7ab0440a758107e204"
format_stamp: "Formatted at 2025-09-05 07:49:19 on dist-test-slave-pgzt"
I20250905 07:49:19.197369 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:49:19.198741 12821 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.199124 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:49:19.199213 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "0e02ae2f2eb74d7ab0440a758107e204"
format_stamp: "Formatted at 2025-09-05 07:49:19 on dist-test-slave-pgzt"
I20250905 07:49:19.199319 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:19.213241 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:19.213725 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:19.214223 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:19.214998 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:19.215055 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.215132 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:19.215308 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.231217 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:33403
I20250905 07:49:19.231240 12883 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:33403 every 8 connection(s)
I20250905 07:49:19.233023 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:19.236182 12888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:19.236537 12889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:19.237748 12891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:19.238026 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:19.238265 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:19.238322 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:19.238377 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058559238366 us; error 0 us; skew 500 ppm
I20250905 07:49:19.238509 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:19.240352 9109 webserver.cc:480] Webserver started at http://127.8.229.69:46087/ using document root <none> and password file <none>
I20250905 07:49:19.240584 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:19.240653 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:19.240783 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:19.241567 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-4-root/instance:
uuid: "af2950144b8c46438eec51827c06eb94"
format_stamp: "Formatted at 2025-09-05 07:49:19 on dist-test-slave-pgzt"
I20250905 07:49:19.241652 12884 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45145
I20250905 07:49:19.241758 12884 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:19.242210 12884 heartbeater.cc:507] Master 127.8.229.126:45145 requested a full tablet report, sending...
I20250905 07:49:19.243072 12360 ts_manager.cc:194] Registered new tserver with Master: 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:19.244040 12360 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37078
I20250905 07:49:19.244446 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250905 07:49:19.246091 12896 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.246414 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:49:19.246510 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-4-root
uuid: "af2950144b8c46438eec51827c06eb94"
format_stamp: "Formatted at 2025-09-05 07:49:19 on dist-test-slave-pgzt"
I20250905 07:49:19.246603 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-4-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-4-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-4-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:19.274433 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:19.275038 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:19.275586 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:19.276387 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:19.276443 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.276543 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:19.276580 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.293298 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.69:42245
I20250905 07:49:19.293334 12958 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.69:42245 every 8 connection(s)
I20250905 07:49:19.294992 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:19.298282 12963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:19.298282 12962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:19.299925 9109 server_base.cc:1047] running on GCE node
W20250905 07:49:19.300041 12965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:19.300390 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:19.300460 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:19.300499 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058559300496 us; error 0 us; skew 500 ppm
I20250905 07:49:19.300738 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:19.301965 9109 webserver.cc:480] Webserver started at http://127.8.229.70:35011/ using document root <none> and password file <none>
I20250905 07:49:19.302157 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:19.302227 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:19.302290 12959 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45145
I20250905 07:49:19.302371 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:19.302407 12959 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:19.302680 12959 heartbeater.cc:507] Master 127.8.229.126:45145 requested a full tablet report, sending...
I20250905 07:49:19.302979 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-5-root/instance:
uuid: "99e007e4f8594df8b4e2aea848ed050c"
format_stamp: "Formatted at 2025-09-05 07:49:19 on dist-test-slave-pgzt"
I20250905 07:49:19.303485 12360 ts_manager.cc:194] Registered new tserver with Master: af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:19.304177 12360 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37080
I20250905 07:49:19.305547 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:49:19.306944 12970 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.307272 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:19.307391 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-5-root
uuid: "99e007e4f8594df8b4e2aea848ed050c"
format_stamp: "Formatted at 2025-09-05 07:49:19 on dist-test-slave-pgzt"
I20250905 07:49:19.307520 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-5-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-5-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-5-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:19.324987 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:19.325598 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:19.326156 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:19.327019 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:19.327080 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.327164 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:19.327201 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:19.341501 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.70:44069
I20250905 07:49:19.341562 13032 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.70:44069 every 8 connection(s)
I20250905 07:49:19.348776 13033 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45145
I20250905 07:49:19.348913 13033 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:19.349349 13033 heartbeater.cc:507] Master 127.8.229.126:45145 requested a full tablet report, sending...
I20250905 07:49:19.350159 12360 ts_manager.cc:194] Registered new tserver with Master: 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069)
I20250905 07:49:19.350916 12360 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37082
I20250905 07:49:19.677026 12465 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:19.679045 12538 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:19.679515 12612 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:19.679598 12808 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:19.680075 12791 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:19.682305 12808 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:19.682760 12612 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:19.682801 12538 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:19.688298 12360 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250905 07:49:19.688984 12430 consensus_peers.cc:489] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e -> Peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403): Couldn't send request to peer 0e02ae2f2eb74d7ab0440a758107e204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3467a871458e47d488238ece84ae086b. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:19.696521 12538 consensus_queue.cc:237] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:19.698645 12612 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:19.699271 12465 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:19.699956 12802 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:19.700537 12802 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:19.702112 13045 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:19.702728 12465 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:19.703298 12612 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
W20250905 07:49:19.706768 12507 consensus_peers.cc:489] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f902eed21ca4a1abaa2335e791c9fd5. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:19.707022 12357 catalog_manager.cc:5582] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:19.710958 12538 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:19.712932 12465 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20250905 07:49:19.713482 12796 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20250905 07:49:19.714236 12612 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20250905 07:49:19.715010 12802 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20250905 07:49:19.715596 12802 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:19.716989 12465 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
W20250905 07:49:19.719976 12508 consensus_peers.cc:489] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f919efd17c674fe799365a7d79d969d1. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:19.720438 12612 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:19.721522 12360 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 28, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:19.722213 12538 consensus_queue.cc:237] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:19.724272 12465 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:19.724817 13048 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20250905 07:49:19.725500 12508 consensus_peers.cc:489] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b380bffe76bd498e8d98e64dae8728ab. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:19.725862 12612 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:19.726264 13048 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:19.727147 13045 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:19.727761 12465 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:19.730090 12614 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:19.733423 12357 catalog_manager.cc:5582] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 29, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:19.733764 12538 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:19.737177 12614 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:19.738188 12802 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:19.740235 12802 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:19.742942 12612 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
W20250905 07:49:19.746063 12505 consensus_peers.cc:489] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403): Couldn't send request to peer 0e02ae2f2eb74d7ab0440a758107e204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e94f73c59d74a2a862c445f8f70e618. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:19.749094 12357 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index -1 to 29, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:19.751296 12465 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:19.751739 12796 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:19.753171 12462 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:19.760569 12612 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:19.762555 12462 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:19.763192 12753 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:19.765704 12753 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:19.766446 12462 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:19.771989 12538 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:19.773095 12357 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:19.778559 13057 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:19.780059 12538 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
W20250905 07:49:19.780908 12581 consensus_peers.cc:489] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 559347a6dcac472c9b09b80a16527175. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:20.046722 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:20.055509 12462 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:20.055711 12462 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:20.061288 12612 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:45306
I20250905 07:49:20.061491 12612 raft_consensus.cc:604] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:20.061951 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250905 07:49:20.062161 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:20.121179 12787 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:20.121384 12787 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:20.121948 12538 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58386
I20250905 07:49:20.122116 12538 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:20.122231 12538 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:20.124107 12538 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:20.124604 12538 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 3 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:20.125078 12612 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:20.125260 12612 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:20.125226 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:20.125411 12462 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:49:20.125496 12462 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:20.125730 12462 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:20.126044 12462 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:20.129777 12462 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 3.
I20250905 07:49:20.129829 12612 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 3.
I20250905 07:49:20.130234 12506 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:20.130497 13045 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:49:20.130628 13045 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:20.130856 13045 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:20.132884 12357 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 2 to 3, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 3 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:20.141235 13059 ts_tablet_manager.cc:927] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: Initiating tablet copy from peer 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:20.142158 13059 tablet_copy_client.cc:323] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:36723
I20250905 07:49:20.148699 12552 tablet_copy_service.cc:140] P 069247fbe8b34bbd81d3549f0fc0b93a: Received BeginTabletCopySession request for tablet 3f902eed21ca4a1abaa2335e791c9fd5 from peer af2950144b8c46438eec51827c06eb94 ({username='slave'} at 127.0.0.1:58428)
I20250905 07:49:20.148866 12552 tablet_copy_service.cc:161] P 069247fbe8b34bbd81d3549f0fc0b93a: Beginning new tablet copy session on tablet 3f902eed21ca4a1abaa2335e791c9fd5 from peer af2950144b8c46438eec51827c06eb94 at {username='slave'} at 127.0.0.1:58428: session id = af2950144b8c46438eec51827c06eb94-3f902eed21ca4a1abaa2335e791c9fd5
I20250905 07:49:20.151523 12552 tablet_copy_source_session.cc:215] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:20.152721 13059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f902eed21ca4a1abaa2335e791c9fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:20.157836 13059 tablet_copy_client.cc:806] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:20.158099 13059 tablet_copy_client.cc:670] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:20.158849 13062 ts_tablet_manager.cc:927] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: Initiating tablet copy from peer 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:20.159524 13062 tablet_copy_client.cc:323] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:34511
I20250905 07:49:20.159627 13060 tablet_copy_client.cc:1025] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:20.164817 12626 tablet_copy_service.cc:140] P 5531c3eeac82412fb18050ea7148b03e: Received BeginTabletCopySession request for tablet 559347a6dcac472c9b09b80a16527175 from peer af2950144b8c46438eec51827c06eb94 ({username='slave'} at 127.0.0.1:45310)
I20250905 07:49:20.164979 12626 tablet_copy_service.cc:161] P 5531c3eeac82412fb18050ea7148b03e: Beginning new tablet copy session on tablet 559347a6dcac472c9b09b80a16527175 from peer af2950144b8c46438eec51827c06eb94 at {username='slave'} at 127.0.0.1:45310: session id = af2950144b8c46438eec51827c06eb94-559347a6dcac472c9b09b80a16527175
I20250905 07:49:20.167621 12626 tablet_copy_source_session.cc:215] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:20.168843 13062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559347a6dcac472c9b09b80a16527175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:20.173756 13062 tablet_copy_client.cc:806] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:20.173961 13062 tablet_copy_client.cc:670] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:20.175397 13063 tablet_copy_client.cc:1025] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:20.194288 13064 ts_tablet_manager.cc:927] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: Initiating tablet copy from peer 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:20.194867 13064 tablet_copy_client.cc:323] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:36723
I20250905 07:49:20.199572 13067 ts_tablet_manager.cc:927] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: Initiating tablet copy from peer 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:20.200616 12552 tablet_copy_service.cc:140] P 069247fbe8b34bbd81d3549f0fc0b93a: Received BeginTabletCopySession request for tablet f919efd17c674fe799365a7d79d969d1 from peer 99e007e4f8594df8b4e2aea848ed050c ({username='slave'} at 127.0.0.1:58430)
I20250905 07:49:20.200625 13067 tablet_copy_client.cc:323] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:36723
I20250905 07:49:20.200819 12552 tablet_copy_service.cc:161] P 069247fbe8b34bbd81d3549f0fc0b93a: Beginning new tablet copy session on tablet f919efd17c674fe799365a7d79d969d1 from peer 99e007e4f8594df8b4e2aea848ed050c at {username='slave'} at 127.0.0.1:58430: session id = 99e007e4f8594df8b4e2aea848ed050c-f919efd17c674fe799365a7d79d969d1
I20250905 07:49:20.203156 12552 tablet_copy_source_session.cc:215] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:20.204460 13064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919efd17c674fe799365a7d79d969d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:20.209084 12552 tablet_copy_service.cc:140] P 069247fbe8b34bbd81d3549f0fc0b93a: Received BeginTabletCopySession request for tablet 0e94f73c59d74a2a862c445f8f70e618 from peer 0e02ae2f2eb74d7ab0440a758107e204 ({username='slave'} at 127.0.0.1:58446)
I20250905 07:49:20.209221 12552 tablet_copy_service.cc:161] P 069247fbe8b34bbd81d3549f0fc0b93a: Beginning new tablet copy session on tablet 0e94f73c59d74a2a862c445f8f70e618 from peer 0e02ae2f2eb74d7ab0440a758107e204 at {username='slave'} at 127.0.0.1:58446: session id = 0e02ae2f2eb74d7ab0440a758107e204-0e94f73c59d74a2a862c445f8f70e618
I20250905 07:49:20.210520 13064 tablet_copy_client.cc:806] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:20.210727 13064 tablet_copy_client.cc:670] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:20.211515 12552 tablet_copy_source_session.cc:215] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:20.212218 13065 tablet_copy_client.cc:1025] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:20.212745 13067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94f73c59d74a2a862c445f8f70e618. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:20.217792 13067 tablet_copy_client.cc:806] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:20.217978 13067 tablet_copy_client.cc:670] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:20.219358 13068 tablet_copy_client.cc:1025] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:20.245545 12884 heartbeater.cc:499] Master 127.8.229.126:45145 was elected leader, sending a full tablet report...
I20250905 07:49:20.275678 13070 ts_tablet_manager.cc:927] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: Initiating tablet copy from peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:20.276383 13070 tablet_copy_client.cc:323] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.65:35247
I20250905 07:49:20.282348 12477 tablet_copy_service.cc:140] P 2df0c6def6514c2a859a74187f5d2d8e: Received BeginTabletCopySession request for tablet 3467a871458e47d488238ece84ae086b from peer 0e02ae2f2eb74d7ab0440a758107e204 ({username='slave'} at 127.0.0.1:59228)
I20250905 07:49:20.282464 12477 tablet_copy_service.cc:161] P 2df0c6def6514c2a859a74187f5d2d8e: Beginning new tablet copy session on tablet 3467a871458e47d488238ece84ae086b from peer 0e02ae2f2eb74d7ab0440a758107e204 at {username='slave'} at 127.0.0.1:59228: session id = 0e02ae2f2eb74d7ab0440a758107e204-3467a871458e47d488238ece84ae086b
I20250905 07:49:20.284830 12477 tablet_copy_source_session.cc:215] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:20.285902 13070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3467a871458e47d488238ece84ae086b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:20.290704 13070 tablet_copy_client.cc:806] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:20.290903 13070 tablet_copy_client.cc:670] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:20.292378 13071 tablet_copy_client.cc:1025] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:20.297582 13072 ts_tablet_manager.cc:927] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: Initiating tablet copy from peer 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:20.298220 13072 tablet_copy_client.cc:323] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:36723
I20250905 07:49:20.299180 12551 tablet_copy_service.cc:140] P 069247fbe8b34bbd81d3549f0fc0b93a: Received BeginTabletCopySession request for tablet b380bffe76bd498e8d98e64dae8728ab from peer 99e007e4f8594df8b4e2aea848ed050c ({username='slave'} at 127.0.0.1:58430)
I20250905 07:49:20.299296 12551 tablet_copy_service.cc:161] P 069247fbe8b34bbd81d3549f0fc0b93a: Beginning new tablet copy session on tablet b380bffe76bd498e8d98e64dae8728ab from peer 99e007e4f8594df8b4e2aea848ed050c at {username='slave'} at 127.0.0.1:58430: session id = 99e007e4f8594df8b4e2aea848ed050c-b380bffe76bd498e8d98e64dae8728ab
I20250905 07:49:20.304023 12551 tablet_copy_source_session.cc:215] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:20.305263 12959 heartbeater.cc:499] Master 127.8.229.126:45145 was elected leader, sending a full tablet report...
I20250905 07:49:20.305794 13072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b380bffe76bd498e8d98e64dae8728ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:20.310912 13072 tablet_copy_client.cc:806] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:20.311103 13072 tablet_copy_client.cc:670] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:20.312706 13074 tablet_copy_client.cc:1025] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:20.352069 13033 heartbeater.cc:499] Master 127.8.229.126:45145 was elected leader, sending a full tablet report...
I20250905 07:49:20.569271 12612 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20250905 07:49:20.570587 13076 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:20.582069 12465 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20250905 07:49:20.582748 13045 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:20.662865 13075 raft_consensus.cc:991] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:20.663055 13075 raft_consensus.cc:1079] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:20.663568 12462 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59192
I20250905 07:49:20.663724 12462 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:20.663815 12462 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:20.666175 12462 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:20.667124 12612 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:20.667348 12612 raft_consensus.cc:3053] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:20.667464 12612 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:20.667719 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:20.667773 12612 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:20.667862 12542 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:20.667994 12612 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:20.669749 12542 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 2.
I20250905 07:49:20.669910 12612 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 2.
I20250905 07:49:20.669991 12462 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:20.670197 12432 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:20.670430 12791 raft_consensus.cc:2802] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:20.670553 12791 raft_consensus.cc:695] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:20.670765 12791 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:20.673516 12359 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 1 to 2, leader changed from 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:21.057303 12538 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:21.057808 12787 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:21.061939 12613 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:21.062635 13087 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:22.062613 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:22.066082 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "5531c3eeac82412fb18050ea7148b03e"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:22.066308 12465 raft_consensus.cc:604] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Received request to transfer leadership to TS 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:22.066725 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:22.067040 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:22.079138 13091 raft_consensus.cc:991] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:22.079308 13091 raft_consensus.cc:1079] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Signalling peer 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:22.079869 12613 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
from {username='slave'} at 127.0.0.1:45276
I20250905 07:49:22.080042 12613 raft_consensus.cc:491] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:22.080152 12613 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:22.082415 12613 raft_consensus.cc:513] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:22.082986 12613 leader_election.cc:290] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 2 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:22.083328 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:22.083498 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:22.083652 12465 raft_consensus.cc:3053] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:22.083725 12465 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:22.083914 12465 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:22.084139 12465 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:22.085951 12465 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 2.
I20250905 07:49:22.083513 12538 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:22.086318 12580 leader_election.cc:304] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:22.086573 13075 raft_consensus.cc:2802] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:22.086712 13075 raft_consensus.cc:695] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:22.086922 13075 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:22.088155 12538 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 2.
I20250905 07:49:22.096499 12357 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 1 to 2, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 2 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:22.159966 13059 tablet_copy_client.cc:538] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:22.163174 13059 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: Bootstrap starting.
I20250905 07:49:22.175738 13062 tablet_copy_client.cc:538] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:22.178628 13062 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: Bootstrap starting.
I20250905 07:49:22.204808 13059 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:22.205329 13059 tablet_bootstrap.cc:492] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: Bootstrap complete.
I20250905 07:49:22.205592 13059 ts_tablet_manager.cc:1397] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: Time spent bootstrapping tablet: real 0.042s user 0.044s sys 0.000s
I20250905 07:49:22.206251 13059 raft_consensus.cc:357] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:22.206411 13059 raft_consensus.cc:738] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: af2950144b8c46438eec51827c06eb94, State: Initialized, Role: LEARNER
I20250905 07:49:22.206615 13059 consensus_queue.cc:260] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:22.207525 13059 ts_tablet_manager.cc:1428] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:22.208248 12551 tablet_copy_service.cc:342] P 069247fbe8b34bbd81d3549f0fc0b93a: Request end of tablet copy session af2950144b8c46438eec51827c06eb94-3f902eed21ca4a1abaa2335e791c9fd5 received from {username='slave'} at 127.0.0.1:58428
I20250905 07:49:22.208379 12551 tablet_copy_service.cc:434] P 069247fbe8b34bbd81d3549f0fc0b93a: ending tablet copy session af2950144b8c46438eec51827c06eb94-3f902eed21ca4a1abaa2335e791c9fd5 on tablet 3f902eed21ca4a1abaa2335e791c9fd5 with peer af2950144b8c46438eec51827c06eb94
I20250905 07:49:22.212524 13064 tablet_copy_client.cc:538] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:22.215370 13064 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap starting.
I20250905 07:49:22.219776 13067 tablet_copy_client.cc:538] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:22.223273 13067 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap starting.
I20250905 07:49:22.227789 13062 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=122 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:22.228283 13062 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: Bootstrap complete.
I20250905 07:49:22.228600 13062 ts_tablet_manager.cc:1397] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: Time spent bootstrapping tablet: real 0.050s user 0.043s sys 0.008s
I20250905 07:49:22.229368 13062 raft_consensus.cc:357] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:22.229652 13062 raft_consensus.cc:738] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: af2950144b8c46438eec51827c06eb94, State: Initialized, Role: LEARNER
I20250905 07:49:22.229908 13062 consensus_queue.cc:260] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:22.230505 13062 ts_tablet_manager.cc:1428] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:49:22.232604 12626 tablet_copy_service.cc:342] P 5531c3eeac82412fb18050ea7148b03e: Request end of tablet copy session af2950144b8c46438eec51827c06eb94-559347a6dcac472c9b09b80a16527175 received from {username='slave'} at 127.0.0.1:45310
I20250905 07:49:22.232720 12626 tablet_copy_service.cc:434] P 5531c3eeac82412fb18050ea7148b03e: ending tablet copy session af2950144b8c46438eec51827c06eb94-559347a6dcac472c9b09b80a16527175 on tablet 559347a6dcac472c9b09b80a16527175 with peer af2950144b8c46438eec51827c06eb94
I20250905 07:49:22.254912 13064 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=99 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:22.255360 13064 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap complete.
I20250905 07:49:22.255656 13064 ts_tablet_manager.cc:1397] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: Time spent bootstrapping tablet: real 0.040s user 0.042s sys 0.000s
I20250905 07:49:22.256395 13064 raft_consensus.cc:357] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:22.256585 13064 raft_consensus.cc:738] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Initialized, Role: LEARNER
I20250905 07:49:22.256798 13064 consensus_queue.cc:260] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:22.258375 13064 ts_tablet_manager.cc:1428] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250905 07:49:22.259471 12551 tablet_copy_service.cc:342] P 069247fbe8b34bbd81d3549f0fc0b93a: Request end of tablet copy session 99e007e4f8594df8b4e2aea848ed050c-f919efd17c674fe799365a7d79d969d1 received from {username='slave'} at 127.0.0.1:58430
I20250905 07:49:22.259580 12551 tablet_copy_service.cc:434] P 069247fbe8b34bbd81d3549f0fc0b93a: ending tablet copy session 99e007e4f8594df8b4e2aea848ed050c-f919efd17c674fe799365a7d79d969d1 on tablet f919efd17c674fe799365a7d79d969d1 with peer 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:22.269181 13067 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=97 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:22.269553 13067 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap complete.
I20250905 07:49:22.269788 13067 ts_tablet_manager.cc:1397] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: Time spent bootstrapping tablet: real 0.047s user 0.035s sys 0.006s
I20250905 07:49:22.270457 13067 raft_consensus.cc:357] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:22.270623 13067 raft_consensus.cc:738] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Initialized, Role: LEARNER
I20250905 07:49:22.270797 13067 consensus_queue.cc:260] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:22.271507 13067 ts_tablet_manager.cc:1428] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20250905 07:49:22.272127 12551 tablet_copy_service.cc:342] P 069247fbe8b34bbd81d3549f0fc0b93a: Request end of tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-0e94f73c59d74a2a862c445f8f70e618 received from {username='slave'} at 127.0.0.1:58446
I20250905 07:49:22.272233 12551 tablet_copy_service.cc:434] P 069247fbe8b34bbd81d3549f0fc0b93a: ending tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-0e94f73c59d74a2a862c445f8f70e618 on tablet 0e94f73c59d74a2a862c445f8f70e618 with peer 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:22.292721 13070 tablet_copy_client.cc:538] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:22.295498 13070 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap starting.
I20250905 07:49:22.313220 13072 tablet_copy_client.cc:538] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:22.316486 13072 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap starting.
I20250905 07:49:22.338819 13070 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=133 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:22.339283 13070 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap complete.
I20250905 07:49:22.339664 13070 ts_tablet_manager.cc:1397] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: Time spent bootstrapping tablet: real 0.044s user 0.046s sys 0.001s
I20250905 07:49:22.340505 13070 raft_consensus.cc:357] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:22.340693 13070 raft_consensus.cc:738] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Initialized, Role: LEARNER
I20250905 07:49:22.340941 13070 consensus_queue.cc:260] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:22.341539 13070 ts_tablet_manager.cc:1428] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:22.342146 12477 tablet_copy_service.cc:342] P 2df0c6def6514c2a859a74187f5d2d8e: Request end of tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-3467a871458e47d488238ece84ae086b received from {username='slave'} at 127.0.0.1:59228
I20250905 07:49:22.342257 12477 tablet_copy_service.cc:434] P 2df0c6def6514c2a859a74187f5d2d8e: ending tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-3467a871458e47d488238ece84ae086b on tablet 3467a871458e47d488238ece84ae086b with peer 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:22.357163 13072 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=106 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:22.357554 13072 tablet_bootstrap.cc:492] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap complete.
I20250905 07:49:22.357784 13072 ts_tablet_manager.cc:1397] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: Time spent bootstrapping tablet: real 0.041s user 0.034s sys 0.005s
I20250905 07:49:22.358384 13072 raft_consensus.cc:357] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:22.358613 13072 raft_consensus.cc:738] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Initialized, Role: LEARNER
I20250905 07:49:22.358810 13072 consensus_queue.cc:260] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:22.359234 13072 ts_tablet_manager.cc:1428] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:49:22.359889 12551 tablet_copy_service.cc:342] P 069247fbe8b34bbd81d3549f0fc0b93a: Request end of tablet copy session 99e007e4f8594df8b4e2aea848ed050c-b380bffe76bd498e8d98e64dae8728ab received from {username='slave'} at 127.0.0.1:58430
I20250905 07:49:22.359987 12551 tablet_copy_service.cc:434] P 069247fbe8b34bbd81d3549f0fc0b93a: ending tablet copy session 99e007e4f8594df8b4e2aea848ed050c-b380bffe76bd498e8d98e64dae8728ab on tablet b380bffe76bd498e8d98e64dae8728ab with peer 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:22.480759 12538 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:22.481253 13075 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:22.488514 12465 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:22.488935 13075 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:22.503944 12934 raft_consensus.cc:1215] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20250905 07:49:22.548208 12859 raft_consensus.cc:1215] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20250905 07:49:22.628564 13008 raft_consensus.cc:1215] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.27->[1.28-1.28] Dedup: 1.28->[]
I20250905 07:49:22.643723 12859 raft_consensus.cc:1215] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20250905 07:49:22.689776 12934 raft_consensus.cc:1215] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20250905 07:49:22.737557 13008 raft_consensus.cc:1215] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20250905 07:49:22.999593 13114 raft_consensus.cc:1062] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: attempting to promote NON_VOTER af2950144b8c46438eec51827c06eb94 to VOTER
I20250905 07:49:23.000453 13114 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:23.002614 12934 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.003154 12462 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.003911 13113 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.004477 12538 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.004724 13113 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.005178 13075 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.008350 13114 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.008955 12462 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.014204 12357 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.014073 12934 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.021296 12613 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:23.024242 12462 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20250905 07:49:23.024834 12934 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20250905 07:49:23.025067 13116 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:23.025333 13114 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:23.026916 13116 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.027861 12465 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.027940 12934 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.030077 12352 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 559347a6dcac472c9b09b80a16527175 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:23.031183 12360 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:23.035058 12526 tablet_service.cc:1515] Processing DeleteTablet for tablet 559347a6dcac472c9b09b80a16527175 with delete_type TABLET_DATA_TOMBSTONED (TS 069247fbe8b34bbd81d3549f0fc0b93a not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:58376
I20250905 07:49:23.035780 13123 tablet_replica.cc:331] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:23.036007 13123 raft_consensus.cc:2241] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:23.036198 13123 pending_rounds.cc:70] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Trying to abort 1 pending ops.
I20250905 07:49:23.036276 13123 pending_rounds.cc:77] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Aborting op as it isn't in flight: id { term: 1 index: 31 } timestamp: 7196911874047754240 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "559347a6dcac472c9b09b80a16527175" old_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } } new_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } } }
I20250905 07:49:23.036823 13123 raft_consensus.cc:2915] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Aborting config change with OpId 1.31: Aborted: Op aborted
I20250905 07:49:23.036940 13123 raft_consensus.cc:2887] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20250905 07:49:23.037048 13123 raft_consensus.cc:2270] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:23.038342 13123 ts_tablet_manager.cc:1905] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:23.043257 13123 ts_tablet_manager.cc:1918] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20250905 07:49:23.043356 13123 log.cc:1199] T 559347a6dcac472c9b09b80a16527175 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/559347a6dcac472c9b09b80a16527175
I20250905 07:49:23.044015 12351 catalog_manager.cc:4928] TS 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): tablet 559347a6dcac472c9b09b80a16527175 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:23.047623 13045 raft_consensus.cc:1062] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: attempting to promote NON_VOTER 99e007e4f8594df8b4e2aea848ed050c to VOTER
I20250905 07:49:23.048462 13045 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:23.050321 13008 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:23.050920 12465 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:23.050931 12613 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:23.051188 13125 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:23.051733 13045 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:23.052244 13045 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:23.055694 13127 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.056494 12613 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.059531 12465 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.059649 13008 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.063337 12357 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 28 to 29, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:23.068411 12542 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:23.069955 13008 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.070678 12465 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.070474 13128 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.001s
I20250905 07:49:23.071568 13125 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:23.074092 13124 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.074676 12465 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.074803 13008 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.077755 12351 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet f919efd17c674fe799365a7d79d969d1 with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:23.079102 12360 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 29 to 30, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:23.082453 12600 tablet_service.cc:1515] Processing DeleteTablet for tablet f919efd17c674fe799365a7d79d969d1 with delete_type TABLET_DATA_TOMBSTONED (TS 5531c3eeac82412fb18050ea7148b03e not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:23.083153 13132 tablet_replica.cc:331] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:23.083340 13132 raft_consensus.cc:2241] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:23.083629 13132 raft_consensus.cc:2270] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:23.084312 13132 ts_tablet_manager.cc:1905] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:23.089390 13132 ts_tablet_manager.cc:1918] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.29
I20250905 07:49:23.089495 13132 log.cc:1199] T f919efd17c674fe799365a7d79d969d1 P 5531c3eeac82412fb18050ea7148b03e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/f919efd17c674fe799365a7d79d969d1
I20250905 07:49:23.090147 12352 catalog_manager.cc:4928] TS 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): tablet f919efd17c674fe799365a7d79d969d1 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:23.096482 13134 raft_consensus.cc:1062] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: attempting to promote NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 to VOTER
I20250905 07:49:23.097308 13134 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:23.099514 12859 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.099524 12613 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.100024 13133 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.100458 13092 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.100603 12538 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.102466 13092 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.105254 13092 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.106063 12538 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.107999 12613 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.110348 12859 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.113178 12359 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.118973 12465 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:23.120860 12859 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20250905 07:49:23.121062 12613 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20250905 07:49:23.121434 13133 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:23.121719 13136 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:23.124145 13134 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.124514 12859 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.124521 12613 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.128603 12350 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 3467a871458e47d488238ece84ae086b with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:23.129042 12360 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 31 to 32, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New cstate: current_term: 1 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.132609 12526 tablet_service.cc:1515] Processing DeleteTablet for tablet 3467a871458e47d488238ece84ae086b with delete_type TABLET_DATA_TOMBSTONED (TS 069247fbe8b34bbd81d3549f0fc0b93a not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:58376
I20250905 07:49:23.132923 13123 tablet_replica.cc:331] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:23.133090 13123 raft_consensus.cc:2241] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:23.133322 13123 raft_consensus.cc:2270] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:23.134141 13123 ts_tablet_manager.cc:1905] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:23.139652 13123 ts_tablet_manager.cc:1918] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20250905 07:49:23.139762 13123 log.cc:1199] T 3467a871458e47d488238ece84ae086b P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/3467a871458e47d488238ece84ae086b
I20250905 07:49:23.140322 12351 catalog_manager.cc:4928] TS 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): tablet 3467a871458e47d488238ece84ae086b (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:23.189654 13125 raft_consensus.cc:1062] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: attempting to promote NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 to VOTER
I20250905 07:49:23.190435 13125 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:23.192720 12859 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.192914 12462 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.193248 12613 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.193230 13128 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:23.193897 13045 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:23.194180 13124 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:23.207223 13124 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.207927 12859 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.211635 12613 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.213081 12365 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 29 to 30, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:23.218556 12538 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:23.220120 12859 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.220139 12613 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.220664 13045 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.221100 13125 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.222870 13045 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.224074 12858 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.227047 12351 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 0e94f73c59d74a2a862c445f8f70e618 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:23.227499 12613 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:23.227747 12357 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 30 to 31, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:23.231348 12448 tablet_service.cc:1515] Processing DeleteTablet for tablet 0e94f73c59d74a2a862c445f8f70e618 with delete_type TABLET_DATA_TOMBSTONED (TS 2df0c6def6514c2a859a74187f5d2d8e not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:23.232074 13144 tablet_replica.cc:331] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:23.232321 13144 raft_consensus.cc:2241] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:23.232569 13144 pending_rounds.cc:70] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Trying to abort 1 pending ops.
I20250905 07:49:23.232678 13144 pending_rounds.cc:77] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Aborting op as it isn't in flight: id { term: 1 index: 30 } timestamp: 7196911874826215424 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "0e94f73c59d74a2a862c445f8f70e618" old_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } } new_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } } }
I20250905 07:49:23.233206 13144 raft_consensus.cc:2915] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Aborting config change with OpId 1.30: Aborted: Op aborted
I20250905 07:49:23.233357 13144 raft_consensus.cc:2887] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20250905 07:49:23.233496 13144 raft_consensus.cc:2270] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:23.234241 13144 ts_tablet_manager.cc:1905] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:23.238987 13144 ts_tablet_manager.cc:1918] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20250905 07:49:23.239122 13144 log.cc:1199] T 0e94f73c59d74a2a862c445f8f70e618 P 2df0c6def6514c2a859a74187f5d2d8e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/0e94f73c59d74a2a862c445f8f70e618
I20250905 07:49:23.239811 12350 catalog_manager.cc:4928] TS 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): tablet 0e94f73c59d74a2a862c445f8f70e618 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:23.267427 13125 raft_consensus.cc:1062] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: attempting to promote NON_VOTER 99e007e4f8594df8b4e2aea848ed050c to VOTER
I20250905 07:49:23.268385 13125 consensus_queue.cc:237] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:23.270390 12613 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.270642 12465 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.270390 13008 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:23.270923 13128 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:23.271142 13045 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:23.271414 13124 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:23.274441 13045 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.274920 13008 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.277988 12465 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.278048 12613 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.278963 12360 catalog_manager.cc:5582] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 29 to 30, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:23.280741 13124 raft_consensus.cc:1062] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: attempting to promote NON_VOTER af2950144b8c46438eec51827c06eb94 to VOTER
I20250905 07:49:23.281528 13124 consensus_queue.cc:237] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:23.284673 12538 consensus_queue.cc:237] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:23.285944 12614 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.286589 12462 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.287099 13124 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.287184 12465 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.287681 13124 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.287907 12934 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.288552 13127 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.289623 13128 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.292204 13127 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.294155 13008 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20250905 07:49:23.294699 13045 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:23.295976 12465 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.296116 13008 raft_consensus.cc:2953] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:23.298194 12357 catalog_manager.cc:5582] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 30 to 31, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:23.300374 12351 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet b380bffe76bd498e8d98e64dae8728ab with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:23.305629 12600 tablet_service.cc:1515] Processing DeleteTablet for tablet b380bffe76bd498e8d98e64dae8728ab with delete_type TABLET_DATA_TOMBSTONED (TS 5531c3eeac82412fb18050ea7148b03e not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:23.305975 13132 tablet_replica.cc:331] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:23.306152 13132 raft_consensus.cc:2241] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:23.306373 13132 raft_consensus.cc:2270] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:23.306648 13127 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.308645 13132 ts_tablet_manager.cc:1905] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:23.308192 12614 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.309967 12462 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.312624 12934 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.314188 12360 catalog_manager.cc:5582] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 30 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:23.316465 13132 ts_tablet_manager.cc:1918] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20250905 07:49:23.316586 13132 log.cc:1199] T b380bffe76bd498e8d98e64dae8728ab P 5531c3eeac82412fb18050ea7148b03e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/b380bffe76bd498e8d98e64dae8728ab
I20250905 07:49:23.317795 12352 catalog_manager.cc:4928] TS 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): tablet b380bffe76bd498e8d98e64dae8728ab (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:23.319639 12538 consensus_queue.cc:237] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:23.321286 12934 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20250905 07:49:23.321727 13124 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:23.321877 12614 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20250905 07:49:23.322285 13124 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:23.323920 13125 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.324579 12934 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.324921 12614 raft_consensus.cc:2953] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.329145 12351 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 3f902eed21ca4a1abaa2335e791c9fd5 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:23.329265 12360 catalog_manager.cc:5582] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 reported cstate change: config changed from index 31 to 32, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New cstate: current_term: 1 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:23.333078 12448 tablet_service.cc:1515] Processing DeleteTablet for tablet 3f902eed21ca4a1abaa2335e791c9fd5 with delete_type TABLET_DATA_TOMBSTONED (TS 2df0c6def6514c2a859a74187f5d2d8e not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:23.333356 13144 tablet_replica.cc:331] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:23.333552 13144 raft_consensus.cc:2241] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:23.333741 13144 raft_consensus.cc:2270] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:23.334558 13144 ts_tablet_manager.cc:1905] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:23.338531 13144 ts_tablet_manager.cc:1918] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20250905 07:49:23.338627 13144 log.cc:1199] T 3f902eed21ca4a1abaa2335e791c9fd5 P 2df0c6def6514c2a859a74187f5d2d8e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/3f902eed21ca4a1abaa2335e791c9fd5
I20250905 07:49:23.339236 12350 catalog_manager.cc:4928] TS 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): tablet 3f902eed21ca4a1abaa2335e791c9fd5 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:24.067414 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:24.070145 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:24.070326 12538 raft_consensus.cc:604] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Received request to transfer leadership to TS 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:24.071272 12462 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "3467a871458e47d488238ece84ae086b"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "5531c3eeac82412fb18050ea7148b03e"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:24.071427 12462 raft_consensus.cc:604] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Received request to transfer leadership to TS 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:24.072293 12614 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:45306
I20250905 07:49:24.072419 12614 raft_consensus.cc:604] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:24.073206 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "af2950144b8c46438eec51827c06eb94"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:24.073362 12538 raft_consensus.cc:604] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Received request to transfer leadership to TS af2950144b8c46438eec51827c06eb94
I20250905 07:49:24.074128 12614 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:45306
I20250905 07:49:24.074266 12614 raft_consensus.cc:604] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:24.075065 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:24.075194 12538 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:24.075562 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20250905 07:49:24.075754 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:24.206619 13124 raft_consensus.cc:991] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:24.206784 13125 raft_consensus.cc:1079] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Signalling peer 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:24.207280 12858 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618"
dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
from {username='slave'} at 127.0.0.1:48310
I20250905 07:49:24.207492 12858 raft_consensus.cc:491] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:24.207585 12858 raft_consensus.cc:3058] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.209591 12858 raft_consensus.cc:513] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:24.210260 12858 leader_election.cc:290] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 2 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:24.211066 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:24.211236 12538 raft_consensus.cc:3053] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:24.211314 12538 raft_consensus.cc:738] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:24.211587 12538 consensus_queue.cc:260] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:24.211927 12538 raft_consensus.cc:3058] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.214130 12538 raft_consensus.cc:2466] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 2.
I20250905 07:49:24.214533 12824 leader_election.cc:304] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 0e02ae2f2eb74d7ab0440a758107e204; no voters:
I20250905 07:49:24.215083 13155 raft_consensus.cc:2802] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:24.215528 13155 raft_consensus.cc:695] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEADER]: Becoming Leader. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Running, Role: LEADER
I20250905 07:49:24.215816 13155 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:24.216606 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0e94f73c59d74a2a862c445f8f70e618" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:24.216774 12614 raft_consensus.cc:3058] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.218043 12360 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 reported cstate change: term changed from 1 to 2, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68). New cstate: current_term: 2 leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:24.219324 12614 raft_consensus.cc:2466] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 2.
I20250905 07:49:24.331645 13124 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:24.331804 13124 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:24.332419 12462 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59180
I20250905 07:49:24.332602 12462 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:24.332700 12462 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:24.335529 12462 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.336195 12462 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 4 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:24.336504 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:24.336587 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:24.336659 12538 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 LEADER]: Stepping down as leader of term 3
I20250905 07:49:24.336733 12538 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:24.336757 12614 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:24.336999 12538 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.337229 12538 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Advancing to term 4
W20250905 07:49:24.338163 12413 auto_rebalancer.cc:254] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
I20250905 07:49:24.339311 12614 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 4.
W20250905 07:49:24.339720 12413 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 5209f5aac7a24f62a3b520b55ae51a28, TS 069247fbe8b34bbd81d3549f0fc0b93a -> TS af2950144b8c46438eec51827c06eb94 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.339706 12538 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 4.
W20250905 07:49:24.339820 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 5209f5aac7a24f62a3b520b55ae51a28, TS 069247fbe8b34bbd81d3549f0fc0b93a -> TS af2950144b8c46438eec51827c06eb94 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.339709 12433 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:24.340284 13133 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Leader election won for term 4
I20250905 07:49:24.340668 13133 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:24.340920 13133 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
W20250905 07:49:24.341159 12413 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 655c91ccdcf4422a8795cecf130a56ed, TS 2df0c6def6514c2a859a74187f5d2d8e -> TS 0e02ae2f2eb74d7ab0440a758107e204 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
W20250905 07:49:24.341215 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 655c91ccdcf4422a8795cecf130a56ed, TS 2df0c6def6514c2a859a74187f5d2d8e -> TS 0e02ae2f2eb74d7ab0440a758107e204 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
W20250905 07:49:24.342650 12413 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 89d6356d89d8404e94394907a2222a62, TS 5531c3eeac82412fb18050ea7148b03e -> TS 99e007e4f8594df8b4e2aea848ed050c move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
W20250905 07:49:24.342730 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 89d6356d89d8404e94394907a2222a62, TS 5531c3eeac82412fb18050ea7148b03e -> TS 99e007e4f8594df8b4e2aea848ed050c move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.343343 12360 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 3 to 4, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 4 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
W20250905 07:49:24.344051 12413 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 3467a871458e47d488238ece84ae086b, TS 5531c3eeac82412fb18050ea7148b03e -> TS af2950144b8c46438eec51827c06eb94 move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
W20250905 07:49:24.344123 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 3467a871458e47d488238ece84ae086b, TS 5531c3eeac82412fb18050ea7148b03e -> TS af2950144b8c46438eec51827c06eb94 move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
W20250905 07:49:24.345347 12413 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 559347a6dcac472c9b09b80a16527175, TS 2df0c6def6514c2a859a74187f5d2d8e -> TS 99e007e4f8594df8b4e2aea848ed050c move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
W20250905 07:49:24.345438 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 559347a6dcac472c9b09b80a16527175, TS 2df0c6def6514c2a859a74187f5d2d8e -> TS 99e007e4f8594df8b4e2aea848ed050c move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
W20250905 07:49:24.346662 12413 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 39d264067d4447c592529d8bf6e39eac, TS 069247fbe8b34bbd81d3549f0fc0b93a -> TS 0e02ae2f2eb74d7ab0440a758107e204 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
W20250905 07:49:24.346727 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 39d264067d4447c592529d8bf6e39eac, TS 069247fbe8b34bbd81d3549f0fc0b93a -> TS 0e02ae2f2eb74d7ab0440a758107e204 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.391265 13125 raft_consensus.cc:991] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower af2950144b8c46438eec51827c06eb94 to start an election
I20250905 07:49:24.391409 13125 raft_consensus.cc:1079] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Signalling peer af2950144b8c46438eec51827c06eb94 to start an election
I20250905 07:49:24.391968 12934 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5"
dest_uuid: "af2950144b8c46438eec51827c06eb94"
from {username='slave'} at 127.0.0.1:37140
I20250905 07:49:24.392131 12934 raft_consensus.cc:491] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:24.392225 12934 raft_consensus.cc:3058] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.394018 12934 raft_consensus.cc:513] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:24.394660 12934 leader_election.cc:290] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [CANDIDATE]: Term 2 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:24.395072 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "af2950144b8c46438eec51827c06eb94" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:24.395072 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3f902eed21ca4a1abaa2335e791c9fd5" candidate_uuid: "af2950144b8c46438eec51827c06eb94" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:24.395278 12614 raft_consensus.cc:3058] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.395278 12538 raft_consensus.cc:3053] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:24.395370 12538 raft_consensus.cc:738] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:24.395612 12538 consensus_queue.cc:260] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:24.395965 12538 raft_consensus.cc:3058] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.398118 12614 raft_consensus.cc:2466] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2950144b8c46438eec51827c06eb94 in term 2.
I20250905 07:49:24.398480 12900 leader_election.cc:304] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5531c3eeac82412fb18050ea7148b03e, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:24.399084 13162 raft_consensus.cc:2802] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:24.399422 13162 raft_consensus.cc:695] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 2 LEADER]: Becoming Leader. State: Replica: af2950144b8c46438eec51827c06eb94, State: Running, Role: LEADER
I20250905 07:49:24.399674 13162 consensus_queue.cc:237] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:24.401901 12360 catalog_manager.cc:5582] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 reported cstate change: term changed from 1 to 2, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to af2950144b8c46438eec51827c06eb94 (127.8.229.69). New cstate: current_term: 2 leader_uuid: "af2950144b8c46438eec51827c06eb94" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:24.406466 12538 raft_consensus.cc:2466] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2950144b8c46438eec51827c06eb94 in term 2.
I20250905 07:49:24.441483 13165 raft_consensus.cc:991] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:24.441726 13116 raft_consensus.cc:1079] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:24.442302 12538 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58378
I20250905 07:49:24.442451 12538 raft_consensus.cc:491] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:24.442518 12538 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.444376 12538 raft_consensus.cc:513] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.444873 12538 leader_election.cc:290] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 2 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:24.445215 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:24.445376 12462 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.445314 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:24.445525 12614 raft_consensus.cc:3053] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:24.445591 12614 raft_consensus.cc:738] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:24.445842 12614 consensus_queue.cc:260] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.446105 12614 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.447377 12462 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 2.
I20250905 07:49:24.447795 12506 leader_election.cc:304] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:24.448038 13125 raft_consensus.cc:2802] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:24.448185 12614 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 2.
I20250905 07:49:24.448179 13125 raft_consensus.cc:695] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:24.448594 13125 consensus_queue.cc:237] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.450920 12360 catalog_manager.cc:5582] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 1 to 2, leader changed from 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:24.531617 13160 raft_consensus.cc:991] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:24.531777 13160 raft_consensus.cc:1079] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Signalling peer 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:24.532387 12614 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "3467a871458e47d488238ece84ae086b"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
from {username='slave'} at 127.0.0.1:45276
I20250905 07:49:24.532555 12614 raft_consensus.cc:491] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:24.532627 12614 raft_consensus.cc:3058] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.534502 12614 raft_consensus.cc:513] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:24.535135 12614 leader_election.cc:290] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 2 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:24.535562 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3467a871458e47d488238ece84ae086b" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:24.535723 12462 raft_consensus.cc:3053] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:24.535787 12462 raft_consensus.cc:738] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:24.535992 12462 consensus_queue.cc:260] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:24.536302 12462 raft_consensus.cc:3058] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.537142 13166 raft_consensus.cc:991] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:24.537263 13166 raft_consensus.cc:1079] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:24.537765 12465 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59192
I20250905 07:49:24.537900 12465 raft_consensus.cc:491] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:24.538004 12465 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:24.538798 12462 raft_consensus.cc:2466] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 2.
I20250905 07:49:24.539145 12580 leader_election.cc:304] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:24.539388 13166 raft_consensus.cc:2802] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:24.539542 13166 raft_consensus.cc:695] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:24.539777 13166 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:24.540338 12465 raft_consensus.cc:513] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.541141 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:24.541294 12538 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:24.542357 12465 leader_election.cc:290] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 3 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:24.542703 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:24.542857 12614 raft_consensus.cc:3053] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:49:24.542884 12858 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3467a871458e47d488238ece84ae086b" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
I20250905 07:49:24.542938 12614 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:24.542809 12360 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 1 to 2, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 2 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:24.543175 12614 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.543421 12614 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:24.543031 12858 raft_consensus.cc:3058] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:24.544622 12538 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 3.
I20250905 07:49:24.545034 12432 leader_election.cc:304] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:24.545341 13160 raft_consensus.cc:2802] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:49:24.545461 13160 raft_consensus.cc:695] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:24.545706 13160 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } }
I20250905 07:49:24.546267 12858 raft_consensus.cc:2466] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 2.
I20250905 07:49:24.547402 12614 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 3.
I20250905 07:49:24.548923 12360 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 2 to 3, leader changed from 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 3 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:24.752332 12538 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:24.752833 13172 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:24.756834 12614 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:24.757387 13172 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:24.770236 12538 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20250905 07:49:24.770747 13160 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:24.774072 12614 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20250905 07:49:24.774552 13160 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:24.918912 12465 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20250905 07:49:24.919420 13125 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:24.922613 12614 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20250905 07:49:24.924100 13184 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:24.927970 12858 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:24.928403 13166 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:24.931330 12465 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:24.931756 13166 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:24.951462 12614 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20250905 07:49:24.951987 13160 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:24.955158 12538 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20250905 07:49:24.955745 13189 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:25.013850 12538 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:25.014374 13191 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:25.018311 12614 raft_consensus.cc:1273] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:25.019126 13191 consensus_queue.cc:1035] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:25.352841 12465 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:25.354826 12614 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:25.355052 12538 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20250905 07:49:25.355676 13133 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:25.355968 13189 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20250905 07:49:25.358664 13189 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:25.359339 12614 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:25.363246 12538 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
W20250905 07:49:25.363747 12432 consensus_peers.cc:489] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39d264067d4447c592529d8bf6e39eac. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:25.364521 12465 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:25.364852 12360 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 29, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New cstate: current_term: 1 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:25.368353 12614 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20250905 07:49:25.368974 13160 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:25.372565 13133 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Committing config change with OpId 3.32: config changed from index -1 to 32, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:25.373005 12538 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20250905 07:49:25.373536 13160 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:25.373469 12614 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index -1 to 32, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
W20250905 07:49:25.374784 12430 consensus_peers.cc:489] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e -> Peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403): Couldn't send request to peer 0e02ae2f2eb74d7ab0440a758107e204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a62c7bcdb1a497898d2078a2f891cff. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:25.374997 12538 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index -1 to 32, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:25.383412 12614 consensus_queue.cc:237] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:25.384378 12365 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 32, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New cstate: current_term: 3 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:25.392993 12465 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:25.393043 12538 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20250905 07:49:25.393801 13116 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:25.394586 13116 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:25.403961 13165 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:25.404860 12538 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:25.405184 12465 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
W20250905 07:49:25.410399 12582 consensus_peers.cc:489] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 655c91ccdcf4422a8795cecf130a56ed. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:25.410202 12360 catalog_manager.cc:5582] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index -1 to 30, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:25.410966 12614 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:25.413628 12934 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20250905 07:49:25.413791 12465 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20250905 07:49:25.414228 13116 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:25.414705 13116 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
W20250905 07:49:25.415522 12582 consensus_peers.cc:489] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 559347a6dcac472c9b09b80a16527175. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:25.417784 13116 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:25.418440 12934 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:25.418440 12465 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:25.422204 12365 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 32 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New cstate: current_term: 1 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:25.422734 12462 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:25.424774 12614 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20250905 07:49:25.426376 13133 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
W20250905 07:49:25.427455 12432 consensus_peers.cc:489] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89d6356d89d8404e94394907a2222a62. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:25.427719 12538 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20250905 07:49:25.428241 13133 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:25.428647 13189 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index -1 to 33, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:25.429145 12614 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index -1 to 33, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:25.432983 12360 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 33, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New cstate: current_term: 4 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:25.433153 12465 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:25.433840 12538 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index -1 to 33, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:25.435197 12542 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
W20250905 07:49:25.435554 12430 consensus_peers.cc:489] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e -> Peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403): Couldn't send request to peer 0e02ae2f2eb74d7ab0440a758107e204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c69a843a1d994abea9c6ec7321f6c3d7. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:25.435918 13133 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:25.436818 12614 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:25.437331 13133 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:25.441107 13160 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index -1 to 31, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:25.441708 12614 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index -1 to 31, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:25.442621 12542 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index -1 to 31, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:25.445796 12357 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 31, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:25.794689 13221 ts_tablet_manager.cc:927] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: Initiating tablet copy from peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:25.795411 13221 tablet_copy_client.cc:323] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.65:35247
I20250905 07:49:25.802275 12477 tablet_copy_service.cc:140] P 2df0c6def6514c2a859a74187f5d2d8e: Received BeginTabletCopySession request for tablet 39d264067d4447c592529d8bf6e39eac from peer af2950144b8c46438eec51827c06eb94 ({username='slave'} at 127.0.0.1:53844)
I20250905 07:49:25.802417 12477 tablet_copy_service.cc:161] P 2df0c6def6514c2a859a74187f5d2d8e: Beginning new tablet copy session on tablet 39d264067d4447c592529d8bf6e39eac from peer af2950144b8c46438eec51827c06eb94 at {username='slave'} at 127.0.0.1:53844: session id = af2950144b8c46438eec51827c06eb94-39d264067d4447c592529d8bf6e39eac
I20250905 07:49:25.804946 12477 tablet_copy_source_session.cc:215] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:25.806102 13221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d264067d4447c592529d8bf6e39eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:25.811293 13221 tablet_copy_client.cc:806] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:25.811524 13221 tablet_copy_client.cc:670] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:25.812876 13222 tablet_copy_client.cc:1025] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:25.819259 13225 ts_tablet_manager.cc:927] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: Initiating tablet copy from peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:25.820098 13225 tablet_copy_client.cc:323] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.65:35247
I20250905 07:49:25.820711 12477 tablet_copy_service.cc:140] P 2df0c6def6514c2a859a74187f5d2d8e: Received BeginTabletCopySession request for tablet 89d6356d89d8404e94394907a2222a62 from peer af2950144b8c46438eec51827c06eb94 ({username='slave'} at 127.0.0.1:53844)
I20250905 07:49:25.820848 12477 tablet_copy_service.cc:161] P 2df0c6def6514c2a859a74187f5d2d8e: Beginning new tablet copy session on tablet 89d6356d89d8404e94394907a2222a62 from peer af2950144b8c46438eec51827c06eb94 at {username='slave'} at 127.0.0.1:53844: session id = af2950144b8c46438eec51827c06eb94-89d6356d89d8404e94394907a2222a62
I20250905 07:49:25.823168 12477 tablet_copy_source_session.cc:215] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:25.824278 13225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d6356d89d8404e94394907a2222a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:25.828761 13225 tablet_copy_client.cc:806] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:25.828997 13225 tablet_copy_client.cc:670] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:25.830497 13226 tablet_copy_client.cc:1025] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:25.886104 13227 ts_tablet_manager.cc:927] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: Initiating tablet copy from peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:25.886740 13227 tablet_copy_client.cc:323] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.65:35247
I20250905 07:49:25.887311 12477 tablet_copy_service.cc:140] P 2df0c6def6514c2a859a74187f5d2d8e: Received BeginTabletCopySession request for tablet 3a62c7bcdb1a497898d2078a2f891cff from peer 0e02ae2f2eb74d7ab0440a758107e204 ({username='slave'} at 127.0.0.1:59228)
I20250905 07:49:25.887501 12477 tablet_copy_service.cc:161] P 2df0c6def6514c2a859a74187f5d2d8e: Beginning new tablet copy session on tablet 3a62c7bcdb1a497898d2078a2f891cff from peer 0e02ae2f2eb74d7ab0440a758107e204 at {username='slave'} at 127.0.0.1:59228: session id = 0e02ae2f2eb74d7ab0440a758107e204-3a62c7bcdb1a497898d2078a2f891cff
I20250905 07:49:25.889861 12477 tablet_copy_source_session.cc:215] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:25.890867 13227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a62c7bcdb1a497898d2078a2f891cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:25.896360 13227 tablet_copy_client.cc:806] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:25.896585 13227 tablet_copy_client.cc:670] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:25.897898 13228 tablet_copy_client.cc:1025] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:25.934084 13229 ts_tablet_manager.cc:927] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: Initiating tablet copy from peer 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:25.934871 13229 tablet_copy_client.cc:323] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:34511
I20250905 07:49:25.945673 12626 tablet_copy_service.cc:140] P 5531c3eeac82412fb18050ea7148b03e: Received BeginTabletCopySession request for tablet 559347a6dcac472c9b09b80a16527175 from peer 99e007e4f8594df8b4e2aea848ed050c ({username='slave'} at 127.0.0.1:54346)
I20250905 07:49:25.945869 12626 tablet_copy_service.cc:161] P 5531c3eeac82412fb18050ea7148b03e: Beginning new tablet copy session on tablet 559347a6dcac472c9b09b80a16527175 from peer 99e007e4f8594df8b4e2aea848ed050c at {username='slave'} at 127.0.0.1:54346: session id = 99e007e4f8594df8b4e2aea848ed050c-559347a6dcac472c9b09b80a16527175
I20250905 07:49:25.948360 12626 tablet_copy_source_session.cc:215] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:25.949692 13234 ts_tablet_manager.cc:927] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: Initiating tablet copy from peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:25.949702 13229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559347a6dcac472c9b09b80a16527175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:25.950284 13234 tablet_copy_client.cc:323] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.65:35247
I20250905 07:49:25.950870 12477 tablet_copy_service.cc:140] P 2df0c6def6514c2a859a74187f5d2d8e: Received BeginTabletCopySession request for tablet c69a843a1d994abea9c6ec7321f6c3d7 from peer 0e02ae2f2eb74d7ab0440a758107e204 ({username='slave'} at 127.0.0.1:59228)
I20250905 07:49:25.951026 12477 tablet_copy_service.cc:161] P 2df0c6def6514c2a859a74187f5d2d8e: Beginning new tablet copy session on tablet c69a843a1d994abea9c6ec7321f6c3d7 from peer 0e02ae2f2eb74d7ab0440a758107e204 at {username='slave'} at 127.0.0.1:59228: session id = 0e02ae2f2eb74d7ab0440a758107e204-c69a843a1d994abea9c6ec7321f6c3d7
I20250905 07:49:25.953497 12477 tablet_copy_source_session.cc:215] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:25.954727 13234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a843a1d994abea9c6ec7321f6c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:25.956200 13229 tablet_copy_client.cc:806] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:25.956454 13229 tablet_copy_client.cc:670] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:25.958037 13231 tablet_copy_client.cc:1025] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:25.960515 13234 tablet_copy_client.cc:806] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:25.960739 13234 tablet_copy_client.cc:670] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:25.962309 13235 tablet_copy_client.cc:1025] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:26.001473 13238 ts_tablet_manager.cc:927] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Initiating tablet copy from peer 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:26.002067 13238 tablet_copy_client.cc:323] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:34511
I20250905 07:49:26.002681 12626 tablet_copy_service.cc:140] P 5531c3eeac82412fb18050ea7148b03e: Received BeginTabletCopySession request for tablet 655c91ccdcf4422a8795cecf130a56ed from peer 99e007e4f8594df8b4e2aea848ed050c ({username='slave'} at 127.0.0.1:54346)
I20250905 07:49:26.002805 12626 tablet_copy_service.cc:161] P 5531c3eeac82412fb18050ea7148b03e: Beginning new tablet copy session on tablet 655c91ccdcf4422a8795cecf130a56ed from peer 99e007e4f8594df8b4e2aea848ed050c at {username='slave'} at 127.0.0.1:54346: session id = 99e007e4f8594df8b4e2aea848ed050c-655c91ccdcf4422a8795cecf130a56ed
I20250905 07:49:26.005267 12626 tablet_copy_source_session.cc:215] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:26.006296 13238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 655c91ccdcf4422a8795cecf130a56ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:26.011356 13238 tablet_copy_client.cc:806] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:26.011622 13238 tablet_copy_client.cc:670] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:26.013309 13239 tablet_copy_client.cc:1025] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:26.076212 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:26.079371 12462 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:26.079653 12462 raft_consensus.cc:604] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:26.080646 12613 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "559347a6dcac472c9b09b80a16527175"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:45306
I20250905 07:49:26.080857 12613 raft_consensus.cc:604] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:26.081954 12462 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:26.082154 12462 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:26.083065 12542 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "99e007e4f8594df8b4e2aea848ed050c"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:26.083252 12542 raft_consensus.cc:604] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Received request to transfer leadership to TS 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:26.083700 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20250905 07:49:26.083949 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:26.120996 13240 raft_consensus.cc:991] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 99e007e4f8594df8b4e2aea848ed050c to start an election
I20250905 07:49:26.121201 13240 raft_consensus.cc:1079] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Signalling peer 99e007e4f8594df8b4e2aea848ed050c to start an election
I20250905 07:49:26.121920 13008 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab"
dest_uuid: "99e007e4f8594df8b4e2aea848ed050c"
from {username='slave'} at 127.0.0.1:52790
I20250905 07:49:26.122152 13008 raft_consensus.cc:491] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:26.122265 13008 raft_consensus.cc:3058] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.124619 13008 raft_consensus.cc:513] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:26.135051 13008 leader_election.cc:290] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [CANDIDATE]: Term 2 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:26.135558 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab" candidate_uuid: "99e007e4f8594df8b4e2aea848ed050c" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:26.135725 12542 raft_consensus.cc:3053] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:26.135789 12542 raft_consensus.cc:738] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:26.136026 12542 consensus_queue.cc:260] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:26.136276 12542 raft_consensus.cc:3058] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.138008 12542 raft_consensus.cc:2466] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e007e4f8594df8b4e2aea848ed050c in term 2.
I20250905 07:49:26.138398 12973 leader_election.cc:304] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 99e007e4f8594df8b4e2aea848ed050c; no voters:
I20250905 07:49:26.138918 13241 raft_consensus.cc:2802] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:26.139276 13241 raft_consensus.cc:695] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEADER]: Becoming Leader. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Running, Role: LEADER
I20250905 07:49:26.139549 13241 consensus_queue.cc:237] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:26.141968 12365 catalog_manager.cc:5582] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c reported cstate change: term changed from 1 to 2, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70). New cstate: current_term: 2 leader_uuid: "99e007e4f8594df8b4e2aea848ed050c" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:26.143828 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b380bffe76bd498e8d98e64dae8728ab" candidate_uuid: "99e007e4f8594df8b4e2aea848ed050c" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:26.143986 12465 raft_consensus.cc:3058] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.146432 12465 raft_consensus.cc:2466] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e007e4f8594df8b4e2aea848ed050c in term 2.
I20250905 07:49:26.351708 13236 raft_consensus.cc:991] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:26.351876 13237 raft_consensus.cc:1079] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:26.352416 12542 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58386
I20250905 07:49:26.352556 12542 raft_consensus.cc:491] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:26.352644 12542 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.355180 12542 raft_consensus.cc:513] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:26.355814 12542 leader_election.cc:290] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 2 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:26.356205 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:26.356204 12613 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:26.356400 12462 raft_consensus.cc:3053] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:26.356427 12613 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.356472 12462 raft_consensus.cc:738] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:26.356705 12462 consensus_queue.cc:260] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:26.357101 12462 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.359079 12613 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 2.
I20250905 07:49:26.359413 12462 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 2.
I20250905 07:49:26.359537 12508 leader_election.cc:304] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:26.359789 13240 raft_consensus.cc:2802] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:26.359894 13240 raft_consensus.cc:695] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:26.360095 13240 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:26.363003 12365 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 1 to 2, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:26.393090 13236 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:26.393230 13237 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:26.393733 12538 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58386
I20250905 07:49:26.393898 12538 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:26.393988 12538 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:26.396504 12538 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:26.397114 12538 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 5 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:26.397452 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:26.397466 12613 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:26.397630 12462 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 LEADER]: Stepping down as leader of term 4
I20250905 07:49:26.397640 12613 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:26.397707 12462 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:26.397958 12462 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:26.398252 12462 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:26.399541 12613 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 5.
I20250905 07:49:26.399875 12508 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:26.400153 13240 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 FOLLOWER]: Leader election won for term 5
I20250905 07:49:26.400282 13240 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:26.400260 12462 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 5.
I20250905 07:49:26.400530 13240 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:26.402889 12365 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 4 to 5, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 5 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:26.435983 13247 raft_consensus.cc:991] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:26.436167 13247 raft_consensus.cc:1079] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:26.436724 12465 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "559347a6dcac472c9b09b80a16527175"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59192
I20250905 07:49:26.436908 12465 raft_consensus.cc:491] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:26.437000 12465 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.439340 12465 raft_consensus.cc:513] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:26.440059 12465 leader_election.cc:290] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Requested vote from peers 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:26.440404 12934 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 33 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:26.440708 12934 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.442833 12934 raft_consensus.cc:2466] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 2.
I20250905 07:49:26.440454 12613 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 33 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:26.443149 12613 raft_consensus.cc:3053] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:26.443212 12613 raft_consensus.cc:738] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:26.443248 12432 leader_election.cc:304] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:26.443508 13237 raft_consensus.cc:2802] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:26.443457 12613 consensus_queue.cc:260] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:26.443611 13237 raft_consensus.cc:695] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:26.443867 12613 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:26.444084 13237 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:26.446813 12613 raft_consensus.cc:2466] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 2.
I20250905 07:49:26.458029 12365 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 1 to 2, leader changed from 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:26.463624 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "559347a6dcac472c9b09b80a16527175"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:26.463779 12465 raft_consensus.cc:604] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Received request to transfer leadership
I20250905 07:49:26.470312 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "559347a6dcac472c9b09b80a16527175"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:26.470475 12465 raft_consensus.cc:604] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Received request to transfer leadership
W20250905 07:49:26.470844 12350 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 559347a6dcac472c9b09b80a16527175 already in progress
I20250905 07:49:26.471999 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "559347a6dcac472c9b09b80a16527175"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:26.472172 12465 raft_consensus.cc:604] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Received request to transfer leadership
W20250905 07:49:26.472872 12413 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 559347a6dcac472c9b09b80a16527175 already in progress
W20250905 07:49:26.472951 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 559347a6dcac472c9b09b80a16527175 already in progress
I20250905 07:49:26.651912 12462 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 99e007e4f8594df8b4e2aea848ed050c: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:26.652395 13250 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:26.658389 12538 raft_consensus.cc:1273] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 99e007e4f8594df8b4e2aea848ed050c: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:26.659932 13250 consensus_queue.cc:1035] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:26.824153 12613 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20250905 07:49:26.824769 13240 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:26.829093 12613 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:26.829645 13236 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:26.831364 12462 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
W20250905 07:49:26.831733 12507 consensus_peers.cc:489] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:26.832376 13220 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
W20250905 07:49:26.832973 12433 consensus_peers.cc:489] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:26.833052 12462 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:26.834471 13220 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:26.847239 13237 raft_consensus.cc:991] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:26.847402 13237 raft_consensus.cc:1079] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Signalling peer 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:26.848328 12612 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "559347a6dcac472c9b09b80a16527175"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
from {username='slave'} at 127.0.0.1:45276
I20250905 07:49:26.849159 12934 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:26.849706 13237 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:26.851200 12612 raft_consensus.cc:491] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:26.851295 12612 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:26.854331 12612 raft_consensus.cc:513] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:26.855367 12612 leader_election.cc:290] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 3 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:26.855691 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 3 candidate_status { last_received { term: 2 index: 34 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:26.857657 12465 raft_consensus.cc:3053] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:49:26.857728 12465 raft_consensus.cc:738] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:26.858011 12465 consensus_queue.cc:260] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:26.858443 12465 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:26.855957 12933 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "559347a6dcac472c9b09b80a16527175" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 3 candidate_status { last_received { term: 2 index: 34 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:26.865494 12465 raft_consensus.cc:2466] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 3.
I20250905 07:49:26.867877 12580 leader_election.cc:304] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters: af2950144b8c46438eec51827c06eb94
I20250905 07:49:26.868295 13247 raft_consensus.cc:2802] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:49:26.868439 13247 raft_consensus.cc:695] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:26.868662 13247 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
W20250905 07:49:26.871075 12507 consensus_peers.cc:489] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:26.873420 12614 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:26.874397 13240 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:26.874387 12357 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 2 to 3, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 3 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250905 07:49:27.297941 12582 consensus_peers.cc:489] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:27.391057 12934 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:27.393915 12934 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20250905 07:49:27.394465 13166 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:27.398379 12462 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20250905 07:49:27.398873 13271 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:27.813230 13221 tablet_copy_client.cc:538] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:27.816668 13221 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: Bootstrap starting.
I20250905 07:49:27.830837 13225 tablet_copy_client.cc:538] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:27.834095 13225 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: Bootstrap starting.
I20250905 07:49:27.854465 13221 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=107 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:27.854876 13221 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: Bootstrap complete.
I20250905 07:49:27.855137 13221 ts_tablet_manager.cc:1397] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: Time spent bootstrapping tablet: real 0.039s user 0.035s sys 0.001s
I20250905 07:49:27.855855 13221 raft_consensus.cc:357] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:27.856019 13221 raft_consensus.cc:738] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: af2950144b8c46438eec51827c06eb94, State: Initialized, Role: LEARNER
I20250905 07:49:27.856228 13221 consensus_queue.cc:260] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:27.856781 13221 ts_tablet_manager.cc:1428] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20250905 07:49:27.857482 12477 tablet_copy_service.cc:342] P 2df0c6def6514c2a859a74187f5d2d8e: Request end of tablet copy session af2950144b8c46438eec51827c06eb94-39d264067d4447c592529d8bf6e39eac received from {username='slave'} at 127.0.0.1:53844
I20250905 07:49:27.857591 12477 tablet_copy_service.cc:434] P 2df0c6def6514c2a859a74187f5d2d8e: ending tablet copy session af2950144b8c46438eec51827c06eb94-39d264067d4447c592529d8bf6e39eac on tablet 39d264067d4447c592529d8bf6e39eac with peer af2950144b8c46438eec51827c06eb94
I20250905 07:49:27.884110 13225 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:27.884536 13225 tablet_bootstrap.cc:492] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: Bootstrap complete.
I20250905 07:49:27.884819 13225 ts_tablet_manager.cc:1397] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: Time spent bootstrapping tablet: real 0.051s user 0.042s sys 0.010s
I20250905 07:49:27.885445 13225 raft_consensus.cc:357] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:27.885594 13225 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: af2950144b8c46438eec51827c06eb94, State: Initialized, Role: LEARNER
I20250905 07:49:27.885793 13225 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:27.886212 13225 ts_tablet_manager.cc:1428] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:49:27.887074 12477 tablet_copy_service.cc:342] P 2df0c6def6514c2a859a74187f5d2d8e: Request end of tablet copy session af2950144b8c46438eec51827c06eb94-89d6356d89d8404e94394907a2222a62 received from {username='slave'} at 127.0.0.1:53844
I20250905 07:49:27.887195 12477 tablet_copy_service.cc:434] P 2df0c6def6514c2a859a74187f5d2d8e: ending tablet copy session af2950144b8c46438eec51827c06eb94-89d6356d89d8404e94394907a2222a62 on tablet 89d6356d89d8404e94394907a2222a62 with peer af2950144b8c46438eec51827c06eb94
I20250905 07:49:27.898236 13227 tablet_copy_client.cc:538] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:27.901316 13227 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap starting.
I20250905 07:49:27.940502 13227 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=122 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:27.940938 13227 tablet_bootstrap.cc:492] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap complete.
I20250905 07:49:27.941226 13227 ts_tablet_manager.cc:1397] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: Time spent bootstrapping tablet: real 0.040s user 0.039s sys 0.001s
I20250905 07:49:27.941965 13227 raft_consensus.cc:357] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:27.942168 13227 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Initialized, Role: LEARNER
I20250905 07:49:27.942365 13227 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:27.942881 13227 ts_tablet_manager.cc:1428] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:49:27.943573 12477 tablet_copy_service.cc:342] P 2df0c6def6514c2a859a74187f5d2d8e: Request end of tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-3a62c7bcdb1a497898d2078a2f891cff received from {username='slave'} at 127.0.0.1:59228
I20250905 07:49:27.943672 12477 tablet_copy_service.cc:434] P 2df0c6def6514c2a859a74187f5d2d8e: ending tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-3a62c7bcdb1a497898d2078a2f891cff on tablet 3a62c7bcdb1a497898d2078a2f891cff with peer 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:27.958449 13229 tablet_copy_client.cc:538] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:27.961679 13229 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap starting.
I20250905 07:49:27.962677 13234 tablet_copy_client.cc:538] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:27.966391 13234 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap starting.
I20250905 07:49:28.006136 13234 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=118 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:28.006639 13234 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap complete.
I20250905 07:49:28.006970 13234 ts_tablet_manager.cc:1397] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: Time spent bootstrapping tablet: real 0.041s user 0.042s sys 0.001s
I20250905 07:49:28.007718 13234 raft_consensus.cc:357] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:28.007896 13234 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Initialized, Role: LEARNER
I20250905 07:49:28.008121 13234 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:28.008710 13234 ts_tablet_manager.cc:1428] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:28.009369 12477 tablet_copy_service.cc:342] P 2df0c6def6514c2a859a74187f5d2d8e: Request end of tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-c69a843a1d994abea9c6ec7321f6c3d7 received from {username='slave'} at 127.0.0.1:59228
I20250905 07:49:28.009466 12477 tablet_copy_service.cc:434] P 2df0c6def6514c2a859a74187f5d2d8e: ending tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-c69a843a1d994abea9c6ec7321f6c3d7 on tablet c69a843a1d994abea9c6ec7321f6c3d7 with peer 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:28.013700 13238 tablet_copy_client.cc:538] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:28.017416 13238 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap starting.
I20250905 07:49:28.019524 13229 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=122 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:28.019938 13229 tablet_bootstrap.cc:492] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap complete.
I20250905 07:49:28.020197 13229 ts_tablet_manager.cc:1397] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: Time spent bootstrapping tablet: real 0.059s user 0.059s sys 0.000s
I20250905 07:49:28.020898 13229 raft_consensus.cc:357] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:28.021157 13229 raft_consensus.cc:738] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Initialized, Role: LEARNER
I20250905 07:49:28.021354 13229 consensus_queue.cc:260] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:28.022055 13229 ts_tablet_manager.cc:1428] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:28.022749 12626 tablet_copy_service.cc:342] P 5531c3eeac82412fb18050ea7148b03e: Request end of tablet copy session 99e007e4f8594df8b4e2aea848ed050c-559347a6dcac472c9b09b80a16527175 received from {username='slave'} at 127.0.0.1:54346
I20250905 07:49:28.022869 12626 tablet_copy_service.cc:434] P 5531c3eeac82412fb18050ea7148b03e: ending tablet copy session 99e007e4f8594df8b4e2aea848ed050c-559347a6dcac472c9b09b80a16527175 on tablet 559347a6dcac472c9b09b80a16527175 with peer 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:28.059607 13238 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=121 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:28.060120 13238 tablet_bootstrap.cc:492] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap complete.
I20250905 07:49:28.060448 13238 ts_tablet_manager.cc:1397] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Time spent bootstrapping tablet: real 0.043s user 0.035s sys 0.008s
I20250905 07:49:28.061072 13238 raft_consensus.cc:357] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:28.061278 13238 raft_consensus.cc:738] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Initialized, Role: LEARNER
I20250905 07:49:28.061463 13238 consensus_queue.cc:260] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:28.061899 13238 ts_tablet_manager.cc:1428] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:49:28.062520 12626 tablet_copy_service.cc:342] P 5531c3eeac82412fb18050ea7148b03e: Request end of tablet copy session 99e007e4f8594df8b4e2aea848ed050c-655c91ccdcf4422a8795cecf130a56ed received from {username='slave'} at 127.0.0.1:54346
I20250905 07:49:28.062628 12626 tablet_copy_service.cc:434] P 5531c3eeac82412fb18050ea7148b03e: ending tablet copy session 99e007e4f8594df8b4e2aea848ed050c-655c91ccdcf4422a8795cecf130a56ed on tablet 655c91ccdcf4422a8795cecf130a56ed with peer 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:28.084262 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:28.087610 12612 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:45306
I20250905 07:49:28.087836 12612 raft_consensus.cc:604] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:28.089107 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:28.089262 12538 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:28.090258 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:28.090433 12538 raft_consensus.cc:604] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:28.090783 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250905 07:49:28.091022 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:28.097425 13271 raft_consensus.cc:991] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:28.097563 13271 raft_consensus.cc:1079] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:28.098125 12538 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58378
I20250905 07:49:28.098290 12538 raft_consensus.cc:491] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:28.098382 12538 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:28.100793 12538 raft_consensus.cc:513] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:28.101318 12538 leader_election.cc:290] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 2 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:28.101725 12612 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:28.101792 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:28.101882 12612 raft_consensus.cc:3053] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:28.101945 12612 raft_consensus.cc:738] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:28.101972 12462 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:28.102180 12612 consensus_queue.cc:260] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:28.102525 12612 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:28.104171 12612 raft_consensus.cc:2466] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 2.
I20250905 07:49:28.104187 12462 raft_consensus.cc:2466] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 2.
I20250905 07:49:28.104542 12508 leader_election.cc:304] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:28.104846 13220 raft_consensus.cc:2802] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:28.105221 13220 raft_consensus.cc:695] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:28.105470 13220 consensus_queue.cc:237] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:28.108199 12357 catalog_manager.cc:5582] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 1 to 2, leader changed from 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:28.124743 12934 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 1 LEARNER]: Advancing to term 2
I20250905 07:49:28.129212 13282 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:28.129364 13282 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:28.129859 12462 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59180
I20250905 07:49:28.130040 12462 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:28.130147 12462 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Advancing to term 6
I20250905 07:49:28.132192 12462 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:28.132894 12462 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 6 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:28.133172 12542 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 34 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:28.133330 12542 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 LEADER]: Stepping down as leader of term 5
I20250905 07:49:28.133280 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 34 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:28.133381 12542 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:28.133441 12614 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Advancing to term 6
I20250905 07:49:28.133610 12542 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 5.34, Last appended by leader: 34, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:28.133976 12542 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 5 FOLLOWER]: Advancing to term 6
I20250905 07:49:28.135588 12614 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 6.
I20250905 07:49:28.135911 12433 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:28.135957 12542 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 6.
I20250905 07:49:28.136143 13237 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 FOLLOWER]: Leader election won for term 6
I20250905 07:49:28.136565 13237 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:28.136811 13237 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 5.34, Last appended by leader: 34, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:28.139084 12357 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 5 to 6, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 6 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:28.173202 12858 raft_consensus.cc:1215] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.31->[3.32-3.32] Dedup: 3.32->[]
I20250905 07:49:28.210500 13220 raft_consensus.cc:991] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:28.210651 13220 raft_consensus.cc:1079] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:28.211153 12462 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59180
I20250905 07:49:28.211349 12462 raft_consensus.cc:491] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:28.211488 12462 raft_consensus.cc:3058] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:28.213594 12462 raft_consensus.cc:513] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:28.214186 12462 leader_election.cc:290] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069)
I20250905 07:49:28.214473 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:28.214560 13008 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f919efd17c674fe799365a7d79d969d1" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "99e007e4f8594df8b4e2aea848ed050c"
I20250905 07:49:28.214736 13008 raft_consensus.cc:3058] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:28.214715 12538 raft_consensus.cc:3053] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:28.214835 12538 raft_consensus.cc:738] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:28.215224 12538 consensus_queue.cc:260] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:28.215588 12538 raft_consensus.cc:3058] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:28.217458 13008 raft_consensus.cc:2466] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 2.
I20250905 07:49:28.217860 12433 leader_election.cc:304] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 99e007e4f8594df8b4e2aea848ed050c; no voters:
I20250905 07:49:28.217947 12538 raft_consensus.cc:2466] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 2.
I20250905 07:49:28.218087 13237 raft_consensus.cc:2802] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:28.218204 13237 raft_consensus.cc:695] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:28.218427 13237 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:28.220847 12357 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 1 to 2, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:28.390952 13008 raft_consensus.cc:3058] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Advancing to term 3
I20250905 07:49:28.393525 13008 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 3 LEARNER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:28.394121 13271 consensus_queue.cc:1037] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:28.404637 13273 raft_consensus.cc:1062] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: attempting to promote NON_VOTER 99e007e4f8594df8b4e2aea848ed050c to VOTER
I20250905 07:49:28.405541 13273 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 3.35, Last appended by leader: 34, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:28.408119 13008 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 3 LEARNER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 3 index: 35. Preceding OpId from leader: term: 3 index: 36. (index mismatch)
I20250905 07:49:28.408581 12462 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 3 index: 35. Preceding OpId from leader: term: 3 index: 36. (index mismatch)
I20250905 07:49:28.408634 12934 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 3 index: 35. Preceding OpId from leader: term: 3 index: 36. (index mismatch)
I20250905 07:49:28.408829 13273 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:28.409098 13271 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:28.409603 13271 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:28.411854 13271 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Committing config change with OpId 3.36: config changed from index 33 to 36, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.412405 12934 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Committing config change with OpId 3.36: config changed from index 33 to 36, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.415488 13008 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Committing config change with OpId 3.36: config changed from index 33 to 36, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.416579 12462 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Committing config change with OpId 3.36: config changed from index 33 to 36, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.417584 12365 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 reported cstate change: config changed from index 33 to 36, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.422132 12614 consensus_queue.cc:237] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 3.36, Last appended by leader: 34, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:28.423902 13008 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 3 index: 36. Preceding OpId from leader: term: 3 index: 37. (index mismatch)
I20250905 07:49:28.424204 12934 raft_consensus.cc:1273] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 3 index: 36. Preceding OpId from leader: term: 3 index: 37. (index mismatch)
I20250905 07:49:28.424427 13296 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20250905 07:49:28.424690 13273 consensus_queue.cc:1035] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20250905 07:49:28.426710 13273 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Committing config change with OpId 3.37: config changed from index 36 to 37, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.427325 13008 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Committing config change with OpId 3.37: config changed from index 36 to 37, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.427675 12934 raft_consensus.cc:2953] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Committing config change with OpId 3.37: config changed from index 36 to 37, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.429862 12352 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 559347a6dcac472c9b09b80a16527175 with cas_config_opid_index 36: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:28.431362 12359 catalog_manager.cc:5582] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 36 to 37, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New cstate: current_term: 3 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:28.434909 12448 tablet_service.cc:1515] Processing DeleteTablet for tablet 559347a6dcac472c9b09b80a16527175 with delete_type TABLET_DATA_TOMBSTONED (TS 2df0c6def6514c2a859a74187f5d2d8e not found in new config with opid_index 37) from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:28.435636 13299 tablet_replica.cc:331] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:28.435914 13299 raft_consensus.cc:2241] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:28.436303 13299 raft_consensus.cc:2270] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:28.437160 13299 ts_tablet_manager.cc:1905] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:28.441524 13299 ts_tablet_manager.cc:1918] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.36
I20250905 07:49:28.441632 13299 log.cc:1199] T 559347a6dcac472c9b09b80a16527175 P 2df0c6def6514c2a859a74187f5d2d8e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/559347a6dcac472c9b09b80a16527175
I20250905 07:49:28.442333 12350 catalog_manager.cc:4928] TS 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): tablet 559347a6dcac472c9b09b80a16527175 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:28.454571 12858 raft_consensus.cc:1215] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.30->[2.31-2.31] Dedup: 2.31->[]
I20250905 07:49:28.483752 12462 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.484241 13220 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.487877 13008 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 1 LEARNER]: Advancing to term 2
I20250905 07:49:28.488193 12614 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.489116 13220 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.490038 13008 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.490780 13220 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.502962 13282 raft_consensus.cc:1062] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: attempting to promote NON_VOTER 99e007e4f8594df8b4e2aea848ed050c to VOTER
I20250905 07:49:28.504159 13282 consensus_queue.cc:237] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 30, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:28.505846 13008 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.506227 12614 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.506419 13302 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.506906 12465 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.507349 13220 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.507529 13282 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.511026 13220 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 30 to 32, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.511550 12462 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 30 to 32, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.511816 12614 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 30 to 32, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.516450 12357 catalog_manager.cc:5582] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 30 to 32, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.525302 13008 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 30 to 32, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.526049 12542 consensus_queue.cc:237] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:28.528172 12614 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:28.528607 13007 raft_consensus.cc:1273] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:28.529067 13220 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:28.529997 12934 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 4 LEARNER]: Advancing to term 6
I20250905 07:49:28.529997 13309 consensus_queue.cc:1035] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:28.532842 12934 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 6 LEARNER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20250905 07:49:28.533057 13220 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.533362 13237 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:28.535248 12614 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.536018 13237 raft_consensus.cc:1062] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: attempting to promote NON_VOTER af2950144b8c46438eec51827c06eb94 to VOTER
W20250905 07:49:28.536136 13237 raft_consensus.cc:1066] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: Unable to promote non-voter af2950144b8c46438eec51827c06eb94: Illegal state: Leader has not yet committed an operation in its own term
I20250905 07:49:28.537011 13008 raft_consensus.cc:2953] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.540944 12359 catalog_manager.cc:5582] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 32 to 33, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:28.543673 12351 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 655c91ccdcf4422a8795cecf130a56ed with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:28.545312 12448 tablet_service.cc:1515] Processing DeleteTablet for tablet 655c91ccdcf4422a8795cecf130a56ed with delete_type TABLET_DATA_TOMBSTONED (TS 2df0c6def6514c2a859a74187f5d2d8e not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:28.545671 13299 tablet_replica.cc:331] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:28.545866 13299 raft_consensus.cc:2241] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:28.546140 13299 raft_consensus.cc:2270] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:28.546972 13299 ts_tablet_manager.cc:1905] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:28.552892 13299 ts_tablet_manager.cc:1918] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.32
I20250905 07:49:28.553025 13299 log.cc:1199] T 655c91ccdcf4422a8795cecf130a56ed P 2df0c6def6514c2a859a74187f5d2d8e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/655c91ccdcf4422a8795cecf130a56ed
I20250905 07:49:28.553774 12350 catalog_manager.cc:4928] TS 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): tablet 655c91ccdcf4422a8795cecf130a56ed (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:28.565831 12538 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20250905 07:49:28.566342 13290 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:28.574723 12614 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 6 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20250905 07:49:28.576514 13312 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:28.584971 13237 raft_consensus.cc:1062] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: attempting to promote NON_VOTER af2950144b8c46438eec51827c06eb94 to VOTER
I20250905 07:49:28.585812 13237 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 6.35, Last appended by leader: 34, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:28.587672 12933 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 6 LEARNER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20250905 07:49:28.587775 12614 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 6 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20250905 07:49:28.588160 13313 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:28.588382 13237 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:28.589295 12538 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20250905 07:49:28.589758 13237 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:28.590632 13313 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 LEADER]: Committing config change with OpId 6.36: config changed from index 33 to 36, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.591187 12614 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 6 FOLLOWER]: Committing config change with OpId 6.36: config changed from index 33 to 36, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.592516 12538 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Committing config change with OpId 6.36: config changed from index 33 to 36, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.595808 12359 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 33 to 36, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.596311 12933 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Committing config change with OpId 6.36: config changed from index 33 to 36, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.601912 12462 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 6.36, Last appended by leader: 34, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:28.603642 12933 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20250905 07:49:28.604219 12538 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20250905 07:49:28.604220 13315 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20250905 07:49:28.604704 13313 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20250905 07:49:28.606549 13312 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 LEADER]: Committing config change with OpId 6.37: config changed from index 36 to 37, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.607170 12538 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Committing config change with OpId 6.37: config changed from index 36 to 37, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.611225 12933 raft_consensus.cc:2953] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Committing config change with OpId 6.37: config changed from index 36 to 37, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.611559 12350 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 89d6356d89d8404e94394907a2222a62 with cas_config_opid_index 36: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:28.611559 12359 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 36 to 37, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New cstate: current_term: 6 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.614645 12600 tablet_service.cc:1515] Processing DeleteTablet for tablet 89d6356d89d8404e94394907a2222a62 with delete_type TABLET_DATA_TOMBSTONED (TS 5531c3eeac82412fb18050ea7148b03e not found in new config with opid_index 37) from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:28.616417 13316 tablet_replica.cc:331] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:28.616650 13316 raft_consensus.cc:2241] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 6 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:28.616922 13316 raft_consensus.cc:2270] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e [term 6 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:28.617923 13316 ts_tablet_manager.cc:1905] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:28.623104 13316 ts_tablet_manager.cc:1918] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.36
I20250905 07:49:28.623224 13316 log.cc:1199] T 89d6356d89d8404e94394907a2222a62 P 5531c3eeac82412fb18050ea7148b03e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/89d6356d89d8404e94394907a2222a62
I20250905 07:49:28.623818 12352 catalog_manager.cc:4928] TS 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): tablet 89d6356d89d8404e94394907a2222a62 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:28.650224 13308 raft_consensus.cc:1062] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: attempting to promote NON_VOTER af2950144b8c46438eec51827c06eb94 to VOTER
I20250905 07:49:28.650857 13308 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:28.652946 12462 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.652954 12933 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.653256 12614 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.653503 13309 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.653707 13307 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.653903 13308 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.656528 13309 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 29 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.659994 12614 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.660854 12933 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.664786 12365 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 reported cstate change: config changed from index 29 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.665848 12465 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.673339 12538 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:28.674909 12465 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.674906 12933 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.675400 13309 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.675670 13308 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.677923 13220 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.678606 12933 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.680306 12465 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:28.681614 12351 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 39d264067d4447c592529d8bf6e39eac with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:28.683172 12365 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:28.686954 12600 tablet_service.cc:1515] Processing DeleteTablet for tablet 39d264067d4447c592529d8bf6e39eac with delete_type TABLET_DATA_TOMBSTONED (TS 5531c3eeac82412fb18050ea7148b03e not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:28.687259 13316 tablet_replica.cc:331] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:28.687515 13316 raft_consensus.cc:2241] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:28.687745 13316 raft_consensus.cc:2270] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:28.688499 13316 ts_tablet_manager.cc:1905] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:28.693460 13316 ts_tablet_manager.cc:1918] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.31
I20250905 07:49:28.693569 13316 log.cc:1199] T 39d264067d4447c592529d8bf6e39eac P 5531c3eeac82412fb18050ea7148b03e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/39d264067d4447c592529d8bf6e39eac
I20250905 07:49:28.694286 12352 catalog_manager.cc:4928] TS 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): tablet 39d264067d4447c592529d8bf6e39eac (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:28.695547 13313 raft_consensus.cc:1062] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: attempting to promote NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 to VOTER
I20250905 07:49:28.696316 13313 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 30, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:28.698104 12614 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20250905 07:49:28.698120 12858 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20250905 07:49:28.698608 13315 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:28.698807 12538 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20250905 07:49:28.698825 13313 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:28.699244 13290 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:28.701906 13315 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Committing config change with OpId 3.33: config changed from index 32 to 33, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.702620 12538 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 32 to 33, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.705482 12858 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 32 to 33, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.706082 12357 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 32 to 33, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:28.717458 12614 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 32 to 33, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.719574 12465 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:28.721819 12858 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20250905 07:49:28.722426 12612 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20250905 07:49:28.722450 13313 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.001s
I20250905 07:49:28.723088 13312 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:28.725519 13008 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.725893 13313 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Committing config change with OpId 3.34: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.726042 12612 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Committing config change with OpId 3.34: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.726265 12858 raft_consensus.cc:2953] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Committing config change with OpId 3.34: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.728049 13315 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.730645 12350 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 3a62c7bcdb1a497898d2078a2f891cff with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:28.731652 12365 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New cstate: current_term: 3 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.734951 12526 tablet_service.cc:1515] Processing DeleteTablet for tablet 3a62c7bcdb1a497898d2078a2f891cff with delete_type TABLET_DATA_TOMBSTONED (TS 069247fbe8b34bbd81d3549f0fc0b93a not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:58376
I20250905 07:49:28.744612 13325 tablet_replica.cc:331] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:28.744807 13325 raft_consensus.cc:2241] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:28.745082 13325 raft_consensus.cc:2270] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:28.747946 13325 ts_tablet_manager.cc:1905] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:28.748296 12538 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:28.748828 13290 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:28.755393 13325 ts_tablet_manager.cc:1918] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.33
I20250905 07:49:28.755527 13325 log.cc:1199] T 3a62c7bcdb1a497898d2078a2f891cff P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/3a62c7bcdb1a497898d2078a2f891cff
I20250905 07:49:28.756158 12351 catalog_manager.cc:4928] TS 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): tablet 3a62c7bcdb1a497898d2078a2f891cff (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:28.893157 13312 raft_consensus.cc:1062] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: attempting to promote NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 to VOTER
I20250905 07:49:28.894016 13312 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:28.896155 12612 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.896281 12858 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.896281 12538 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:28.896823 13312 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.897118 13313 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.897614 13313 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:28.912631 13312 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.913368 12858 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.917285 12357 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 31 to 32, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:28.917695 12538 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.916472 12612 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.921464 12462 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:28.923590 12858 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:28.924028 13313 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:28.926568 13315 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.926970 12614 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:28.927904 12858 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.928238 13312 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:28.929858 12612 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:28.930891 12350 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet c69a843a1d994abea9c6ec7321f6c3d7 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:28.932302 12357 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 32 to 33, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:28.936421 12526 tablet_service.cc:1515] Processing DeleteTablet for tablet c69a843a1d994abea9c6ec7321f6c3d7 with delete_type TABLET_DATA_TOMBSTONED (TS 069247fbe8b34bbd81d3549f0fc0b93a not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:58376
I20250905 07:49:28.936740 13325 tablet_replica.cc:331] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:28.936919 13325 raft_consensus.cc:2241] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:28.937199 13325 raft_consensus.cc:2270] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:28.937975 13325 ts_tablet_manager.cc:1905] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:28.942323 13325 ts_tablet_manager.cc:1918] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.32
I20250905 07:49:28.942436 13325 log.cc:1199] T c69a843a1d994abea9c6ec7321f6c3d7 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/c69a843a1d994abea9c6ec7321f6c3d7
I20250905 07:49:28.943111 12351 catalog_manager.cc:4928] TS 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): tablet c69a843a1d994abea9c6ec7321f6c3d7 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:29.937510 12462 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:29.939721 13008 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:29.939728 12538 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
W20250905 07:49:29.940121 12432 consensus_peers.cc:489] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f919efd17c674fe799365a7d79d969d1. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:29.940197 13313 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:29.940424 13312 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:29.942744 13315 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 30 to 32, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:29.943249 12538 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 30 to 32, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:29.943249 13008 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 30 to 32, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:29.946981 12359 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 30 to 32, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:29.947391 12542 consensus_queue.cc:237] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
W20250905 07:49:29.952702 12505 consensus_peers.cc:489] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403): Couldn't send request to peer 0e02ae2f2eb74d7ab0440a758107e204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5209f5aac7a24f62a3b520b55ae51a28. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:29.953356 12462 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:29.953866 13309 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:29.956305 13309 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:29.956722 12462 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:29.966881 12614 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20250905 07:49:29.967008 12612 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:29.967407 13341 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20250905 07:49:29.968835 12357 catalog_manager.cc:5582] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:29.969218 12614 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:29.971700 12462 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
W20250905 07:49:29.972280 12582 consensus_peers.cc:489] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3467a871458e47d488238ece84ae086b. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:29.972429 13271 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:29.976272 13271 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:29.976946 12859 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:29.977433 13347 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:29.978996 12859 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:29.979820 12462 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:29.981175 12357 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 32 to 34, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New cstate: current_term: 2 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:29.985201 12542 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:29.987259 12933 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:29.987510 12462 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
W20250905 07:49:29.987555 12505 consensus_peers.cc:489] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403): Couldn't send request to peer 0e02ae2f2eb74d7ab0440a758107e204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39d264067d4447c592529d8bf6e39eac. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:29.987872 13336 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:29.988087 13309 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:29.990484 13309 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:29.991080 12462 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:29.995640 12357 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 32 to 33, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:29.995679 12933 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } } }
I20250905 07:49:29.996457 12462 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:29.999775 12859 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:30.000176 12612 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:30.000319 13315 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
W20250905 07:49:30.000559 12432 consensus_peers.cc:489] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c69a843a1d994abea9c6ec7321f6c3d7. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:30.000804 13315 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:30.008960 13313 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:30.009498 12612 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:30.009649 12859 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:30.013957 12365 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 33 to 34, NON_VOTER af2950144b8c46438eec51827c06eb94 (127.8.229.69) added. New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } } }
I20250905 07:49:30.030496 12859 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:30.033280 12542 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:30.033582 12612 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:30.033854 13358 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:30.034206 13199 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:30.037564 13199 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:30.038285 12612 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
W20250905 07:49:30.040858 12825 consensus_peers.cc:489] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 -> Peer 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70:44069): Couldn't send request to peer 99e007e4f8594df8b4e2aea848ed050c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e94f73c59d74a2a862c445f8f70e618. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:30.041569 12542 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } }
I20250905 07:49:30.043377 12357 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 reported cstate change: config changed from index 31 to 33, NON_VOTER 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) added. New cstate: current_term: 2 leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:30.091367 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:30.094996 12542 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "af2950144b8c46438eec51827c06eb94"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:30.095307 12542 raft_consensus.cc:604] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Received request to transfer leadership to TS af2950144b8c46438eec51827c06eb94
I20250905 07:49:30.096431 12462 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:30.096681 12462 raft_consensus.cc:604] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Received request to transfer leadership to TS 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:30.097702 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "5531c3eeac82412fb18050ea7148b03e"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:30.097842 12465 raft_consensus.cc:604] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Received request to transfer leadership to TS 5531c3eeac82412fb18050ea7148b03e
I20250905 07:49:30.098182 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250905 07:49:30.098397 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:30.169337 13313 raft_consensus.cc:991] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:30.169502 13313 raft_consensus.cc:1079] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Signalling peer 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:30.169996 12859 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
from {username='slave'} at 127.0.0.1:48294
I20250905 07:49:30.170161 12859 raft_consensus.cc:491] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:30.170255 12859 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:30.172619 12859 raft_consensus.cc:513] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:30.173357 12859 leader_election.cc:290] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 4 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:30.173712 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:30.173750 12612 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:30.173902 12462 raft_consensus.cc:3053] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Stepping down as leader of term 3
I20250905 07:49:30.173933 12612 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:30.173966 12462 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:30.174161 12462 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 3.34, Last appended by leader: 34, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:30.174381 12462 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:30.175976 12612 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 4.
I20250905 07:49:30.176237 12462 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 4.
I20250905 07:49:30.176371 12825 leader_election.cc:304] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02ae2f2eb74d7ab0440a758107e204, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:30.176642 13199 raft_consensus.cc:2802] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Leader election won for term 4
I20250905 07:49:30.176775 13199 raft_consensus.cc:695] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Becoming Leader. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Running, Role: LEADER
I20250905 07:49:30.177070 13199 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 3.34, Last appended by leader: 34, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:30.179504 12365 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 reported cstate change: term changed from 3 to 4, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68). New cstate: current_term: 4 leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:30.373018 13336 raft_consensus.cc:991] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower af2950144b8c46438eec51827c06eb94 to start an election
I20250905 07:49:30.373242 13309 raft_consensus.cc:1079] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Signalling peer af2950144b8c46438eec51827c06eb94 to start an election
I20250905 07:49:30.373822 12933 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac"
dest_uuid: "af2950144b8c46438eec51827c06eb94"
from {username='slave'} at 127.0.0.1:37140
I20250905 07:49:30.374034 12933 raft_consensus.cc:491] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:30.375856 13365 ts_tablet_manager.cc:927] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: Initiating tablet copy from peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247)
I20250905 07:49:30.376467 13365 tablet_copy_client.cc:323] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.65:35247
I20250905 07:49:30.377086 12477 tablet_copy_service.cc:140] P 2df0c6def6514c2a859a74187f5d2d8e: Received BeginTabletCopySession request for tablet f919efd17c674fe799365a7d79d969d1 from peer af2950144b8c46438eec51827c06eb94 ({username='slave'} at 127.0.0.1:53844)
I20250905 07:49:30.377213 12477 tablet_copy_service.cc:161] P 2df0c6def6514c2a859a74187f5d2d8e: Beginning new tablet copy session on tablet f919efd17c674fe799365a7d79d969d1 from peer af2950144b8c46438eec51827c06eb94 at {username='slave'} at 127.0.0.1:53844: session id = af2950144b8c46438eec51827c06eb94-f919efd17c674fe799365a7d79d969d1
I20250905 07:49:30.379746 12477 tablet_copy_source_session.cc:215] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:30.382369 13365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919efd17c674fe799365a7d79d969d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:30.389084 13365 tablet_copy_client.cc:806] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:30.389417 13365 tablet_copy_client.cc:670] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:30.390841 13366 tablet_copy_client.cc:1025] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:30.396132 12933 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:30.398237 12933 raft_consensus.cc:513] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:30.398983 12933 leader_election.cc:290] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [CANDIDATE]: Term 3 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:30.399477 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "af2950144b8c46438eec51827c06eb94" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:30.399495 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "39d264067d4447c592529d8bf6e39eac" candidate_uuid: "af2950144b8c46438eec51827c06eb94" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:30.399662 12462 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:30.399690 12538 raft_consensus.cc:3053] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:49:30.399763 12538 raft_consensus.cc:738] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:30.400038 12538 consensus_queue.cc:260] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:30.400482 12538 raft_consensus.cc:3058] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:30.402266 12462 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2950144b8c46438eec51827c06eb94 in term 3.
I20250905 07:49:30.402645 12898 leader_election.cc:304] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:30.402771 12538 raft_consensus.cc:2466] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2950144b8c46438eec51827c06eb94 in term 3.
I20250905 07:49:30.402925 13193 raft_consensus.cc:2802] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:49:30.403527 13193 raft_consensus.cc:695] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 3 LEADER]: Becoming Leader. State: Replica: af2950144b8c46438eec51827c06eb94, State: Running, Role: LEADER
I20250905 07:49:30.403796 13193 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:30.406339 12365 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 reported cstate change: term changed from 2 to 3, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to af2950144b8c46438eec51827c06eb94 (127.8.229.69). New cstate: current_term: 3 leader_uuid: "af2950144b8c46438eec51827c06eb94" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:30.492098 13369 ts_tablet_manager.cc:927] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: Initiating tablet copy from peer 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:30.492734 13369 tablet_copy_client.cc:323] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:34511
I20250905 07:49:30.493484 12626 tablet_copy_service.cc:140] P 5531c3eeac82412fb18050ea7148b03e: Received BeginTabletCopySession request for tablet 3467a871458e47d488238ece84ae086b from peer 99e007e4f8594df8b4e2aea848ed050c ({username='slave'} at 127.0.0.1:54346)
I20250905 07:49:30.493626 12626 tablet_copy_service.cc:161] P 5531c3eeac82412fb18050ea7148b03e: Beginning new tablet copy session on tablet 3467a871458e47d488238ece84ae086b from peer 99e007e4f8594df8b4e2aea848ed050c at {username='slave'} at 127.0.0.1:54346: session id = 99e007e4f8594df8b4e2aea848ed050c-3467a871458e47d488238ece84ae086b
I20250905 07:49:30.495980 13371 ts_tablet_manager.cc:927] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: Initiating tablet copy from peer 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723)
I20250905 07:49:30.496034 12626 tablet_copy_source_session.cc:215] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:30.496747 13371 tablet_copy_client.cc:323] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:36723
I20250905 07:49:30.497287 12551 tablet_copy_service.cc:140] P 069247fbe8b34bbd81d3549f0fc0b93a: Received BeginTabletCopySession request for tablet 5209f5aac7a24f62a3b520b55ae51a28 from peer 0e02ae2f2eb74d7ab0440a758107e204 ({username='slave'} at 127.0.0.1:58446)
I20250905 07:49:30.497412 13369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3467a871458e47d488238ece84ae086b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:30.497424 12551 tablet_copy_service.cc:161] P 069247fbe8b34bbd81d3549f0fc0b93a: Beginning new tablet copy session on tablet 5209f5aac7a24f62a3b520b55ae51a28 from peer 0e02ae2f2eb74d7ab0440a758107e204 at {username='slave'} at 127.0.0.1:58446: session id = 0e02ae2f2eb74d7ab0440a758107e204-5209f5aac7a24f62a3b520b55ae51a28
I20250905 07:49:30.500198 12551 tablet_copy_source_session.cc:215] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:30.501376 13371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5209f5aac7a24f62a3b520b55ae51a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:30.503633 13369 tablet_copy_client.cc:806] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:30.503829 13369 tablet_copy_client.cc:670] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:30.505429 13370 tablet_copy_client.cc:1025] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:30.507897 13371 tablet_copy_client.cc:806] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:30.508121 13371 tablet_copy_client.cc:670] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:30.509479 13372 tablet_copy_client.cc:1025] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:30.514302 13313 raft_consensus.cc:991] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:30.514441 13313 raft_consensus.cc:1079] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Signalling peer 5531c3eeac82412fb18050ea7148b03e to start an election
I20250905 07:49:30.514943 12612 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
from {username='slave'} at 127.0.0.1:45276
I20250905 07:49:30.515110 12612 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:30.515209 12612 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:30.517745 12612 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:30.518404 12612 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 3 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:30.518641 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 3 candidate_status { last_received { term: 2 index: 34 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:30.518811 12462 raft_consensus.cc:3053] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:49:30.518882 12462 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:30.519227 12859 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "5531c3eeac82412fb18050ea7148b03e" candidate_term: 3 candidate_status { last_received { term: 2 index: 34 } } ignore_live_leader: true dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
I20250905 07:49:30.519546 12859 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:30.521296 12859 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 3.
I20250905 07:49:30.521682 12579 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02ae2f2eb74d7ab0440a758107e204, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:30.521749 13373 ts_tablet_manager.cc:927] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: Initiating tablet copy from peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:30.519140 12462 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:30.521971 13347 raft_consensus.cc:2802] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:49:30.522094 13347 raft_consensus.cc:695] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Becoming Leader. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:30.522158 12462 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:30.522389 13347 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:30.524715 12462 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5531c3eeac82412fb18050ea7148b03e in term 3.
I20250905 07:49:30.525178 13373 tablet_copy_client.cc:323] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.68:33403
I20250905 07:49:30.525183 12365 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: term changed from 2 to 3, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 5531c3eeac82412fb18050ea7148b03e (127.8.229.67). New cstate: current_term: 3 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:30.531986 12612 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:30.532006 12869 tablet_copy_service.cc:140] P 0e02ae2f2eb74d7ab0440a758107e204: Received BeginTabletCopySession request for tablet 0e94f73c59d74a2a862c445f8f70e618 from peer 99e007e4f8594df8b4e2aea848ed050c ({username='slave'} at 127.0.0.1:49624)
I20250905 07:49:30.532182 12612 raft_consensus.cc:604] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Received request to transfer leadership
I20250905 07:49:30.532191 12869 tablet_copy_service.cc:161] P 0e02ae2f2eb74d7ab0440a758107e204: Beginning new tablet copy session on tablet 0e94f73c59d74a2a862c445f8f70e618 from peer 99e007e4f8594df8b4e2aea848ed050c at {username='slave'} at 127.0.0.1:49624: session id = 99e007e4f8594df8b4e2aea848ed050c-0e94f73c59d74a2a862c445f8f70e618
I20250905 07:49:30.534525 12869 tablet_copy_source_session.cc:215] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:30.535584 13373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94f73c59d74a2a862c445f8f70e618. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:30.540238 12612 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:30.540441 12612 raft_consensus.cc:604] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Received request to transfer leadership
I20250905 07:49:30.540956 13373 tablet_copy_client.cc:806] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:30.541222 13373 tablet_copy_client.cc:670] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:30.541731 12612 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:30.541849 12612 raft_consensus.cc:604] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Received request to transfer leadership
W20250905 07:49:30.542205 12413 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for c69a843a1d994abea9c6ec7321f6c3d7 already in progress
W20250905 07:49:30.542274 12413 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for c69a843a1d994abea9c6ec7321f6c3d7 already in progress
I20250905 07:49:30.542701 13374 tablet_copy_client.cc:1025] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:30.590720 12612 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20250905 07:49:30.591270 13199 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:30.605930 12462 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20250905 07:49:30.606487 13199 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:30.796077 12542 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20250905 07:49:30.796566 13193 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:30.800151 12462 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20250905 07:49:30.801257 13367 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
W20250905 07:49:30.808806 12897 consensus_peers.cc:489] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 -> Peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403): Couldn't send request to peer 0e02ae2f2eb74d7ab0440a758107e204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39d264067d4447c592529d8bf6e39eac. This is attempt 1: this message will repeat every 5th retry.
W20250905 07:49:30.910468 12581 consensus_peers.cc:489] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c69a843a1d994abea9c6ec7321f6c3d7. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:30.995882 12858 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20250905 07:49:30.996443 13347 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:30.998359 13271 raft_consensus.cc:991] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: : Instructing follower 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:30.998589 13347 raft_consensus.cc:1079] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Signalling peer 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:30.999425 12859 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
from {username='slave'} at 127.0.0.1:49614
I20250905 07:49:30.999624 12859 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:30.999725 12859 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:31.002532 12859 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:31.003531 12462 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 4 candidate_status { last_received { term: 3 index: 35 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:31.003592 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 4 candidate_status { last_received { term: 3 index: 35 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:31.003721 12614 raft_consensus.cc:3053] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Stepping down as leader of term 3
I20250905 07:49:31.003794 12614 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5531c3eeac82412fb18050ea7148b03e, State: Running, Role: LEADER
I20250905 07:49:31.004020 12614 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 3.35, Last appended by leader: 35, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:31.004251 12614 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:31.003527 12859 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 4 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:31.006392 12614 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 4.
I20250905 07:49:31.003721 12462 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:31.006721 12825 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02ae2f2eb74d7ab0440a758107e204, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:31.006960 13199 raft_consensus.cc:2802] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Leader election won for term 4
I20250905 07:49:31.007104 13199 raft_consensus.cc:695] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Becoming Leader. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Running, Role: LEADER
I20250905 07:49:31.007467 13199 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 3.35, Last appended by leader: 35, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:31.009935 12462 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 4.
I20250905 07:49:31.010756 12365 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 reported cstate change: term changed from 3 to 4, leader changed from 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) to 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68). New cstate: current_term: 4 leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:31.352217 13390 ts_tablet_manager.cc:927] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: Initiating tablet copy from peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:31.352813 13390 tablet_copy_client.cc:323] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.69:42245
I20250905 07:49:31.358290 12944 tablet_copy_service.cc:140] P af2950144b8c46438eec51827c06eb94: Received BeginTabletCopySession request for tablet 39d264067d4447c592529d8bf6e39eac from peer 0e02ae2f2eb74d7ab0440a758107e204 ({username='slave'} at 127.0.0.1:46116)
I20250905 07:49:31.358424 12944 tablet_copy_service.cc:161] P af2950144b8c46438eec51827c06eb94: Beginning new tablet copy session on tablet 39d264067d4447c592529d8bf6e39eac from peer 0e02ae2f2eb74d7ab0440a758107e204 at {username='slave'} at 127.0.0.1:46116: session id = 0e02ae2f2eb74d7ab0440a758107e204-39d264067d4447c592529d8bf6e39eac
I20250905 07:49:31.360743 12944 tablet_copy_source_session.cc:215] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:31.361891 13390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d264067d4447c592529d8bf6e39eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:31.367362 13390 tablet_copy_client.cc:806] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:31.367578 13390 tablet_copy_client.cc:670] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:31.369037 13391 tablet_copy_client.cc:1025] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Injecting latency into file download: 2000
W20250905 07:49:31.403148 12824 consensus_peers.cc:489] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c69a843a1d994abea9c6ec7321f6c3d7. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:31.463388 12614 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 3 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20250905 07:49:31.463922 13389 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:31.467876 12462 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20250905 07:49:31.468382 13389 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:31.862076 13400 ts_tablet_manager.cc:927] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: Initiating tablet copy from peer 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:31.862643 13400 tablet_copy_client.cc:323] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.68:33403
I20250905 07:49:31.863161 12869 tablet_copy_service.cc:140] P 0e02ae2f2eb74d7ab0440a758107e204: Received BeginTabletCopySession request for tablet c69a843a1d994abea9c6ec7321f6c3d7 from peer af2950144b8c46438eec51827c06eb94 ({username='slave'} at 127.0.0.1:49634)
I20250905 07:49:31.863269 12869 tablet_copy_service.cc:161] P 0e02ae2f2eb74d7ab0440a758107e204: Beginning new tablet copy session on tablet c69a843a1d994abea9c6ec7321f6c3d7 from peer af2950144b8c46438eec51827c06eb94 at {username='slave'} at 127.0.0.1:49634: session id = af2950144b8c46438eec51827c06eb94-c69a843a1d994abea9c6ec7321f6c3d7
I20250905 07:49:31.865561 12869 tablet_copy_source_session.cc:215] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:31.866526 13400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a843a1d994abea9c6ec7321f6c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:31.871719 13400 tablet_copy_client.cc:806] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:31.871941 13400 tablet_copy_client.cc:670] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:31.873194 13401 tablet_copy_client.cc:1025] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: tablet copy: Injecting latency into file download: 2000
I20250905 07:49:32.098724 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:32.107833 12859 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:50800
I20250905 07:49:32.108050 12859 raft_consensus.cc:604] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:32.108503 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:32.108757 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:32.391193 13365 tablet_copy_client.cc:538] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:32.394217 13365 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: Bootstrap starting.
I20250905 07:49:32.428812 13365 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=99 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:32.429183 13365 tablet_bootstrap.cc:492] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: Bootstrap complete.
I20250905 07:49:32.429461 13365 ts_tablet_manager.cc:1397] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: Time spent bootstrapping tablet: real 0.035s user 0.030s sys 0.005s
I20250905 07:49:32.430032 13365 raft_consensus.cc:357] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:32.430235 13365 raft_consensus.cc:738] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: af2950144b8c46438eec51827c06eb94, State: Initialized, Role: LEARNER
I20250905 07:49:32.430446 13365 consensus_queue.cc:260] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:32.430827 13365 ts_tablet_manager.cc:1428] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:49:32.431380 12477 tablet_copy_service.cc:342] P 2df0c6def6514c2a859a74187f5d2d8e: Request end of tablet copy session af2950144b8c46438eec51827c06eb94-f919efd17c674fe799365a7d79d969d1 received from {username='slave'} at 127.0.0.1:53844
I20250905 07:49:32.431499 12477 tablet_copy_service.cc:434] P 2df0c6def6514c2a859a74187f5d2d8e: ending tablet copy session af2950144b8c46438eec51827c06eb94-f919efd17c674fe799365a7d79d969d1 on tablet f919efd17c674fe799365a7d79d969d1 with peer af2950144b8c46438eec51827c06eb94
I20250905 07:49:32.443428 12934 raft_consensus.cc:1215] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.31->[2.32-2.32] Dedup: 2.32->[]
I20250905 07:49:32.472995 13389 raft_consensus.cc:991] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:32.473160 13389 raft_consensus.cc:1079] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:32.473691 12465 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59228
I20250905 07:49:32.473863 12465 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:32.473973 12465 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:32.476258 12465 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:32.477025 12465 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 5 election: Requested vote from peers 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511), 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:32.477378 12614 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 36 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:32.477378 12859 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 36 } } ignore_live_leader: true dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
I20250905 07:49:32.477581 12859 raft_consensus.cc:3053] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Stepping down as leader of term 4
I20250905 07:49:32.477581 12614 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:32.477664 12859 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Running, Role: LEADER
I20250905 07:49:32.477895 12859 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 4.36, Last appended by leader: 36, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:32.478255 12859 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:32.480248 12614 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 5.
I20250905 07:49:32.480665 12433 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:32.480947 13387 raft_consensus.cc:2802] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Leader election won for term 5
I20250905 07:49:32.480671 12859 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 5.
I20250905 07:49:32.481446 13387 raft_consensus.cc:695] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:32.481696 13387 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 4.36, Last appended by leader: 36, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:32.483974 12365 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 4 to 5, leader changed from 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 5 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:32.505734 13369 tablet_copy_client.cc:538] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:32.508648 13369 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap starting.
I20250905 07:49:32.510906 13371 tablet_copy_client.cc:538] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:32.514186 13371 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap starting.
I20250905 07:49:32.543022 13373 tablet_copy_client.cc:538] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:32.546445 13373 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap starting.
I20250905 07:49:32.561623 13371 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=111 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:32.562160 13371 tablet_bootstrap.cc:492] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap complete.
I20250905 07:49:32.562570 13371 ts_tablet_manager.cc:1397] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: Time spent bootstrapping tablet: real 0.048s user 0.041s sys 0.006s
I20250905 07:49:32.563362 13371 raft_consensus.cc:357] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:32.563728 13371 raft_consensus.cc:738] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Initialized, Role: LEARNER
I20250905 07:49:32.564002 13371 consensus_queue.cc:260] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:32.566716 13371 ts_tablet_manager.cc:1428] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250905 07:49:32.568123 12551 tablet_copy_service.cc:342] P 069247fbe8b34bbd81d3549f0fc0b93a: Request end of tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-5209f5aac7a24f62a3b520b55ae51a28 received from {username='slave'} at 127.0.0.1:58446
I20250905 07:49:32.568266 12551 tablet_copy_service.cc:434] P 069247fbe8b34bbd81d3549f0fc0b93a: ending tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-5209f5aac7a24f62a3b520b55ae51a28 on tablet 5209f5aac7a24f62a3b520b55ae51a28 with peer 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:32.569682 13369 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=133 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:32.570057 13369 tablet_bootstrap.cc:492] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap complete.
I20250905 07:49:32.570325 13369 ts_tablet_manager.cc:1397] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: Time spent bootstrapping tablet: real 0.062s user 0.046s sys 0.001s
I20250905 07:49:32.571009 13369 raft_consensus.cc:357] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:32.571213 13369 raft_consensus.cc:738] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Initialized, Role: LEARNER
I20250905 07:49:32.571420 13369 consensus_queue.cc:260] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:32.572245 13369 ts_tablet_manager.cc:1428] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20250905 07:49:32.572825 12626 tablet_copy_service.cc:342] P 5531c3eeac82412fb18050ea7148b03e: Request end of tablet copy session 99e007e4f8594df8b4e2aea848ed050c-3467a871458e47d488238ece84ae086b received from {username='slave'} at 127.0.0.1:54346
I20250905 07:49:32.572942 12626 tablet_copy_service.cc:434] P 5531c3eeac82412fb18050ea7148b03e: ending tablet copy session 99e007e4f8594df8b4e2aea848ed050c-3467a871458e47d488238ece84ae086b on tablet 3467a871458e47d488238ece84ae086b with peer 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:32.593165 13373 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=97 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:32.593595 13373 tablet_bootstrap.cc:492] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: Bootstrap complete.
I20250905 07:49:32.593870 13373 ts_tablet_manager.cc:1397] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: Time spent bootstrapping tablet: real 0.047s user 0.034s sys 0.012s
I20250905 07:49:32.594458 13373 raft_consensus.cc:357] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:32.594666 13373 raft_consensus.cc:738] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Initialized, Role: LEARNER
I20250905 07:49:32.594889 13373 consensus_queue.cc:260] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } }
I20250905 07:49:32.595377 13373 ts_tablet_manager.cc:1428] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20250905 07:49:32.595994 12869 tablet_copy_service.cc:342] P 0e02ae2f2eb74d7ab0440a758107e204: Request end of tablet copy session 99e007e4f8594df8b4e2aea848ed050c-0e94f73c59d74a2a862c445f8f70e618 received from {username='slave'} at 127.0.0.1:49624
I20250905 07:49:32.596144 12869 tablet_copy_service.cc:434] P 0e02ae2f2eb74d7ab0440a758107e204: ending tablet copy session 99e007e4f8594df8b4e2aea848ed050c-0e94f73c59d74a2a862c445f8f70e618 on tablet 0e94f73c59d74a2a862c445f8f70e618 with peer 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:32.760123 12859 raft_consensus.cc:1215] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.29->[2.30-2.30] Dedup: 2.30->[]
W20250905 07:49:32.905436 12432 consensus_peers.cc:489] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e -> Peer af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245): Couldn't send request to peer af2950144b8c46438eec51827c06eb94. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:32.920657 12612 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 5 index: 37. (index mismatch)
I20250905 07:49:32.921176 13387 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20250905 07:49:32.926477 12859 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 5 index: 37. (index mismatch)
I20250905 07:49:32.927495 13406 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20250905 07:49:32.960165 13417 raft_consensus.cc:1062] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: attempting to promote NON_VOTER af2950144b8c46438eec51827c06eb94 to VOTER
I20250905 07:49:32.960879 13417 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 30, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:32.962970 13008 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:32.963383 12934 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 LEARNER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:32.963498 12542 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20250905 07:49:32.963488 13387 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:32.964313 13417 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:32.964543 13406 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:32.968111 13417 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.968698 13008 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.968828 12934 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.971239 12542 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.974117 12359 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c reported cstate change: config changed from index 32 to 33, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.979314 12465 consensus_queue.cc:237] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:32.981334 12934 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:32.981514 13008 raft_consensus.cc:1273] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:32.981876 13387 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:32.982118 13406 consensus_queue.cc:1035] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:32.984298 13417 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.984818 13008 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.985083 12933 raft_consensus.cc:2953] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:32.987645 12350 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet f919efd17c674fe799365a7d79d969d1 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:32.988752 12365 catalog_manager.cc:5582] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 33 to 34, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New cstate: current_term: 2 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:32.992231 12526 tablet_service.cc:1515] Processing DeleteTablet for tablet f919efd17c674fe799365a7d79d969d1 with delete_type TABLET_DATA_TOMBSTONED (TS 069247fbe8b34bbd81d3549f0fc0b93a not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:58376
I20250905 07:49:32.993013 13424 tablet_replica.cc:331] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:32.993227 13424 raft_consensus.cc:2241] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:32.993533 13424 raft_consensus.cc:2270] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:32.994415 13424 ts_tablet_manager.cc:1905] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:32.995853 13008 raft_consensus.cc:1215] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.33->[2.34-2.34] Dedup: 2.34->[]
I20250905 07:49:32.999712 13424 ts_tablet_manager.cc:1918] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.33
I20250905 07:49:32.999850 13424 log.cc:1199] T f919efd17c674fe799365a7d79d969d1 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/f919efd17c674fe799365a7d79d969d1
I20250905 07:49:33.000535 12351 catalog_manager.cc:4928] TS 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): tablet f919efd17c674fe799365a7d79d969d1 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:33.022017 13008 raft_consensus.cc:1215] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.32->[2.33-2.33] Dedup: 2.33->[]
I20250905 07:49:33.227137 13426 raft_consensus.cc:1062] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: attempting to promote NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 to VOTER
I20250905 07:49:33.228045 13426 consensus_queue.cc:237] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 28, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:33.230413 12858 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEARNER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:33.230600 12612 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:33.230929 13426 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:33.231043 12462 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20250905 07:49:33.231245 13397 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:33.231933 13397 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20250905 07:49:33.234850 13426 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:33.235720 12462 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:33.239041 12612 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:33.239204 12859 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:33.239998 12357 catalog_manager.cc:5582] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 30 to 31, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:33.246528 12538 consensus_queue.cc:237] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:33.248275 12462 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:33.248466 12859 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20250905 07:49:33.248735 13425 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:33.249055 13397 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20250905 07:49:33.251636 12858 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:33.251752 13425 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:33.252913 12465 raft_consensus.cc:2953] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:33.255031 12351 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 5209f5aac7a24f62a3b520b55ae51a28 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:33.255857 12357 catalog_manager.cc:5582] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: config changed from index 31 to 32, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New cstate: current_term: 2 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:33.258970 12600 tablet_service.cc:1515] Processing DeleteTablet for tablet 5209f5aac7a24f62a3b520b55ae51a28 with delete_type TABLET_DATA_TOMBSTONED (TS 5531c3eeac82412fb18050ea7148b03e not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:33.259796 13431 tablet_replica.cc:331] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:33.260279 13431 raft_consensus.cc:2241] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:33.260499 13431 raft_consensus.cc:2270] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:33.261258 13431 ts_tablet_manager.cc:1905] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:33.265338 13431 ts_tablet_manager.cc:1918] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.31
I20250905 07:49:33.265421 13431 log.cc:1199] T 5209f5aac7a24f62a3b520b55ae51a28 P 5531c3eeac82412fb18050ea7148b03e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/5209f5aac7a24f62a3b520b55ae51a28
I20250905 07:49:33.265988 12352 catalog_manager.cc:4928] TS 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): tablet 5209f5aac7a24f62a3b520b55ae51a28 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:33.369408 13390 tablet_copy_client.cc:538] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:33.372455 13390 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap starting.
I20250905 07:49:33.408473 13390 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=107 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:33.408895 13390 tablet_bootstrap.cc:492] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: Bootstrap complete.
I20250905 07:49:33.409132 13390 ts_tablet_manager.cc:1397] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: Time spent bootstrapping tablet: real 0.037s user 0.032s sys 0.004s
I20250905 07:49:33.409778 13390 raft_consensus.cc:357] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:33.409940 13390 raft_consensus.cc:738] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Initialized, Role: LEARNER
I20250905 07:49:33.410109 13390 consensus_queue.cc:260] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 3.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: true } }
I20250905 07:49:33.410574 13390 ts_tablet_manager.cc:1428] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:49:33.411130 12944 tablet_copy_service.cc:342] P af2950144b8c46438eec51827c06eb94: Request end of tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-39d264067d4447c592529d8bf6e39eac received from {username='slave'} at 127.0.0.1:46116
I20250905 07:49:33.411240 12944 tablet_copy_service.cc:434] P af2950144b8c46438eec51827c06eb94: ending tablet copy session 0e02ae2f2eb74d7ab0440a758107e204-39d264067d4447c592529d8bf6e39eac on tablet 39d264067d4447c592529d8bf6e39eac with peer 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:33.417567 13433 raft_consensus.cc:1062] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: attempting to promote NON_VOTER 99e007e4f8594df8b4e2aea848ed050c to VOTER
I20250905 07:49:33.418293 13433 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 31, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:33.420564 12538 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:33.420593 12612 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:33.420998 13008 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20250905 07:49:33.421029 13389 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:33.421247 13434 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:33.421733 13434 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:33.424902 13389 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.425442 13008 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.430160 12365 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c reported cstate change: config changed from index 33 to 34, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.435976 12858 consensus_queue.cc:237] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:33.436197 12614 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.440152 12614 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20250905 07:49:33.440838 13008 raft_consensus.cc:1273] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20250905 07:49:33.441561 13440 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:33.441849 13433 consensus_queue.cc:1035] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:33.443459 13440 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.444141 13008 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.446909 12408 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 0e94f73c59d74a2a862c445f8f70e618 with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:33.448472 12357 catalog_manager.cc:5582] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 reported cstate change: config changed from index 34 to 35, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New cstate: current_term: 2 leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:33.448185 12614 raft_consensus.cc:2953] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.451680 12526 tablet_service.cc:1515] Processing DeleteTablet for tablet 0e94f73c59d74a2a862c445f8f70e618 with delete_type TABLET_DATA_TOMBSTONED (TS 069247fbe8b34bbd81d3549f0fc0b93a not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:58376
I20250905 07:49:33.452196 13424 tablet_replica.cc:331] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:33.452378 13424 raft_consensus.cc:2241] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:33.452559 13424 pending_rounds.cc:70] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Trying to abort 1 pending ops.
I20250905 07:49:33.452612 13424 pending_rounds.cc:77] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Aborting op as it isn't in flight: id { term: 2 index: 34 } timestamp: 7196911916719738880 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "0e94f73c59d74a2a862c445f8f70e618" old_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: NON_VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: true } } } new_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } } }
I20250905 07:49:33.452852 13424 raft_consensus.cc:2915] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Aborting config change with OpId 2.34: Aborted: Op aborted
I20250905 07:49:33.452939 13424 raft_consensus.cc:2887] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20250905 07:49:33.453006 13424 raft_consensus.cc:2270] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:33.453735 13424 ts_tablet_manager.cc:1905] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:33.458951 13424 ts_tablet_manager.cc:1918] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.34
I20250905 07:49:33.459086 13424 log.cc:1199] T 0e94f73c59d74a2a862c445f8f70e618 P 069247fbe8b34bbd81d3549f0fc0b93a: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/0e94f73c59d74a2a862c445f8f70e618
I20250905 07:49:33.459656 12351 catalog_manager.cc:4928] TS 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723): tablet 0e94f73c59d74a2a862c445f8f70e618 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:33.469985 13442 raft_consensus.cc:1062] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: attempting to promote NON_VOTER 99e007e4f8594df8b4e2aea848ed050c to VOTER
I20250905 07:49:33.470866 13442 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 32, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:33.472729 12858 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20250905 07:49:33.473150 12465 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20250905 07:49:33.473143 13441 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:33.473325 13008 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEARNER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20250905 07:49:33.474246 13441 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.001s
I20250905 07:49:33.474561 13399 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:33.481240 13399 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.482568 12465 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.485668 13008 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.485754 12365 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 34 to 35, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:33.486369 12858 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.493124 12614 consensus_queue.cc:237] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 2.35, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:33.494618 13008 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20250905 07:49:33.494728 12858 raft_consensus.cc:1273] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Refusing update from remote peer 5531c3eeac82412fb18050ea7148b03e: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20250905 07:49:33.495165 13444 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:33.495452 13442 consensus_queue.cc:1035] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:33.497293 13441 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.497876 12858 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.497876 13008 raft_consensus.cc:2953] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } } }
I20250905 07:49:33.500527 12352 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 3467a871458e47d488238ece84ae086b with cas_config_opid_index 35: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:33.502163 12357 catalog_manager.cc:5582] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New cstate: current_term: 2 leader_uuid: "5531c3eeac82412fb18050ea7148b03e" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:33.505857 12448 tablet_service.cc:1515] Processing DeleteTablet for tablet 3467a871458e47d488238ece84ae086b with delete_type TABLET_DATA_TOMBSTONED (TS 2df0c6def6514c2a859a74187f5d2d8e not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:33.506480 13448 tablet_replica.cc:331] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:33.506728 13448 raft_consensus.cc:2241] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:33.506981 13448 raft_consensus.cc:2270] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:33.507970 13448 ts_tablet_manager.cc:1905] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:33.513949 13448 ts_tablet_manager.cc:1918] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.35
I20250905 07:49:33.514155 13448 log.cc:1199] T 3467a871458e47d488238ece84ae086b P 2df0c6def6514c2a859a74187f5d2d8e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/3467a871458e47d488238ece84ae086b
I20250905 07:49:33.514828 12350 catalog_manager.cc:4928] TS 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): tablet 3467a871458e47d488238ece84ae086b (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:33.704826 12858 raft_consensus.cc:1215] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Deduplicated request from leader. Original: 2.33->[3.34-3.34] Dedup: 3.34->[]
I20250905 07:49:33.873512 13400 tablet_copy_client.cc:538] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:33.876513 13400 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: Bootstrap starting.
I20250905 07:49:33.922164 13400 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=118 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:33.922662 13400 tablet_bootstrap.cc:492] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: Bootstrap complete.
I20250905 07:49:33.923015 13400 ts_tablet_manager.cc:1397] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: Time spent bootstrapping tablet: real 0.046s user 0.043s sys 0.000s
I20250905 07:49:33.923885 13400 raft_consensus.cc:357] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:33.924124 13400 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: af2950144b8c46438eec51827c06eb94, State: Initialized, Role: LEARNER
I20250905 07:49:33.924338 13400 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 4.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: NON_VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: true } }
I20250905 07:49:33.924914 13400 ts_tablet_manager.cc:1428] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:49:33.925696 12869 tablet_copy_service.cc:342] P 0e02ae2f2eb74d7ab0440a758107e204: Request end of tablet copy session af2950144b8c46438eec51827c06eb94-c69a843a1d994abea9c6ec7321f6c3d7 received from {username='slave'} at 127.0.0.1:49634
I20250905 07:49:33.925820 12869 tablet_copy_service.cc:434] P 0e02ae2f2eb74d7ab0440a758107e204: ending tablet copy session af2950144b8c46438eec51827c06eb94-c69a843a1d994abea9c6ec7321f6c3d7 on tablet c69a843a1d994abea9c6ec7321f6c3d7 with peer af2950144b8c46438eec51827c06eb94
I20250905 07:49:34.027189 12933 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 4 LEARNER]: Advancing to term 5
I20250905 07:49:34.109153 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:34.111987 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:34.112164 12465 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:34.112582 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:34.112777 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:34.127614 13454 raft_consensus.cc:1062] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: attempting to promote NON_VOTER 0e02ae2f2eb74d7ab0440a758107e204 to VOTER
I20250905 07:49:34.128551 13454 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 3.34, Last appended by leader: 33, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:34.130510 12858 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 LEARNER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20250905 07:49:34.130528 12538 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20250905 07:49:34.130848 12465 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20250905 07:49:34.131052 13457 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:34.131352 13454 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:34.131630 13404 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20250905 07:49:34.144397 13450 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:34.144623 13450 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:34.145201 12542 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58386
I20250905 07:49:34.145380 12542 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:34.145474 12542 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 6 FOLLOWER]: Advancing to term 7
I20250905 07:49:34.146310 13454 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 3 LEADER]: Committing config change with OpId 3.35: config changed from index 33 to 35, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:34.147814 12542 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:34.148326 12542 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 7 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:34.148783 12934 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:34.148943 12934 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Advancing to term 7
I20250905 07:49:34.150147 12538 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Committing config change with OpId 3.35: config changed from index 33 to 35, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:34.150708 12365 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 reported cstate change: config changed from index 33 to 35, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "af2950144b8c46438eec51827c06eb94" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:34.151532 12934 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 7.
I20250905 07:49:34.151908 12507 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:34.151711 12462 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Committing config change with OpId 3.35: config changed from index 33 to 35, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:34.152221 13397 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 FOLLOWER]: Leader election won for term 7
I20250905 07:49:34.156570 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:34.156291 12858 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Committing config change with OpId 3.35: config changed from index 33 to 35, 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } attrs { replace: true } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:34.156966 12465 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 LEADER]: Stepping down as leader of term 6
I20250905 07:49:34.157027 12465 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:34.157255 12465 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.37, Last appended by leader: 37, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:34.157543 12465 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 FOLLOWER]: Advancing to term 7
I20250905 07:49:34.159989 12465 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 7.
I20250905 07:49:34.165055 13397 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:34.165367 13397 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.37, Last appended by leader: 37, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:34.168226 12360 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 6 to 7, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 7 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:34.170694 12933 consensus_queue.cc:237] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 3.35, Last appended by leader: 33, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:34.172595 12538 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 3 index: 35. Preceding OpId from leader: term: 3 index: 36. (index mismatch)
I20250905 07:49:34.172658 12858 raft_consensus.cc:1273] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Refusing update from remote peer af2950144b8c46438eec51827c06eb94: Log matching property violated. Preceding OpId in replica: term: 3 index: 35. Preceding OpId from leader: term: 3 index: 36. (index mismatch)
I20250905 07:49:34.173285 13455 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:34.173619 13404 consensus_queue.cc:1035] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:34.175580 13454 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 3 LEADER]: Committing config change with OpId 3.36: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:34.176275 12538 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Committing config change with OpId 3.36: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:34.176385 12858 raft_consensus.cc:2953] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Committing config change with OpId 3.36: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } }
I20250905 07:49:34.179044 12411 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 39d264067d4447c592529d8bf6e39eac with cas_config_opid_index 35: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:34.180171 12357 catalog_manager.cc:5582] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 reported cstate change: config changed from index 35 to 36, VOTER 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) evicted. New cstate: current_term: 3 leader_uuid: "af2950144b8c46438eec51827c06eb94" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:34.182960 12448 tablet_service.cc:1515] Processing DeleteTablet for tablet 39d264067d4447c592529d8bf6e39eac with delete_type TABLET_DATA_TOMBSTONED (TS 2df0c6def6514c2a859a74187f5d2d8e not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:59170
I20250905 07:49:34.183595 13468 tablet_replica.cc:331] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:34.183769 13468 raft_consensus.cc:2241] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:34.184018 13468 raft_consensus.cc:2270] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:34.184728 13468 ts_tablet_manager.cc:1905] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:34.188519 13468 ts_tablet_manager.cc:1918] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.35
I20250905 07:49:34.188618 13468 log.cc:1199] T 39d264067d4447c592529d8bf6e39eac P 2df0c6def6514c2a859a74187f5d2d8e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/39d264067d4447c592529d8bf6e39eac
I20250905 07:49:34.189157 12350 catalog_manager.cc:4928] TS 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): tablet 39d264067d4447c592529d8bf6e39eac (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:34.492199 13461 raft_consensus.cc:1062] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: attempting to promote NON_VOTER af2950144b8c46438eec51827c06eb94 to VOTER
I20250905 07:49:34.492985 13461 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 5.37, Last appended by leader: 36, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:34.495025 12934 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 5 LEARNER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 37. Preceding OpId from leader: term: 5 index: 38. (index mismatch)
I20250905 07:49:34.495064 12858 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 37. Preceding OpId from leader: term: 5 index: 38. (index mismatch)
I20250905 07:49:34.495481 13469 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20250905 07:49:34.495780 12612 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 37. Preceding OpId from leader: term: 5 index: 38. (index mismatch)
I20250905 07:49:34.495756 13471 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20250905 07:49:34.496228 13469 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20250905 07:49:34.498556 13471 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Committing config change with OpId 5.38: config changed from index 34 to 38, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.499150 12612 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Committing config change with OpId 5.38: config changed from index 34 to 38, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.502666 12934 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 5 FOLLOWER]: Committing config change with OpId 5.38: config changed from index 34 to 38, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.504966 12360 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e reported cstate change: config changed from index 34 to 38, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.513006 12858 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Committing config change with OpId 5.38: config changed from index 34 to 38, af2950144b8c46438eec51827c06eb94 (127.8.229.69) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } attrs { replace: true } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.515029 12465 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 5.38, Last appended by leader: 36, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:34.516644 12934 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 38. Preceding OpId from leader: term: 5 index: 39. (index mismatch)
I20250905 07:49:34.517117 13469 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20250905 07:49:34.518015 12858 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 38. Preceding OpId from leader: term: 5 index: 39. (index mismatch)
I20250905 07:49:34.518456 13469 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20250905 07:49:34.519170 13461 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Committing config change with OpId 5.39: config changed from index 38 to 39, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.519698 12934 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 5 FOLLOWER]: Committing config change with OpId 5.39: config changed from index 38 to 39, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.521988 12350 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet c69a843a1d994abea9c6ec7321f6c3d7 with cas_config_opid_index 38: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:34.522943 12357 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: config changed from index 38 to 39, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New cstate: current_term: 5 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:49:34.524204 12858 raft_consensus.cc:2953] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Committing config change with OpId 5.39: config changed from index 38 to 39, VOTER 5531c3eeac82412fb18050ea7148b03e (127.8.229.67) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } } }
I20250905 07:49:34.526141 12600 tablet_service.cc:1515] Processing DeleteTablet for tablet c69a843a1d994abea9c6ec7321f6c3d7 with delete_type TABLET_DATA_TOMBSTONED (TS 5531c3eeac82412fb18050ea7148b03e not found in new config with opid_index 39) from {username='slave'} at 127.0.0.1:45250
I20250905 07:49:34.526731 13474 tablet_replica.cc:331] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:34.526919 13474 raft_consensus.cc:2241] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:34.527209 13474 raft_consensus.cc:2270] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:34.528854 13474 ts_tablet_manager.cc:1905] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:34.532773 13474 ts_tablet_manager.cc:1918] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.38
I20250905 07:49:34.532868 13474 log.cc:1199] T c69a843a1d994abea9c6ec7321f6c3d7 P 5531c3eeac82412fb18050ea7148b03e: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/c69a843a1d994abea9c6ec7321f6c3d7
I20250905 07:49:34.533473 12352 catalog_manager.cc:4928] TS 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511): tablet c69a843a1d994abea9c6ec7321f6c3d7 (table test-workload [id=4bcb011aec8c49849ee77cf19ee8f8c3]) successfully deleted
I20250905 07:49:34.577904 12934 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 7 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 6 index: 37. Preceding OpId from leader: term: 7 index: 38. (index mismatch)
I20250905 07:49:34.578465 13397 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20250905 07:49:34.582005 12465 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 7 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 6 index: 37. Preceding OpId from leader: term: 7 index: 38. (index mismatch)
I20250905 07:49:34.582588 13464 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20250905 07:49:36.113133 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:36.115726 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:36.115923 12538 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:36.116377 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:36.116544 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:36.537339 13397 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:36.537523 13397 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:36.537976 12465 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59180
I20250905 07:49:36.538121 12465 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:36.538208 12465 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 7 FOLLOWER]: Advancing to term 8
I20250905 07:49:36.540127 12465 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:36.540591 12465 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 8 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:36.540975 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 38 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:36.541121 12538 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 LEADER]: Stepping down as leader of term 7
I20250905 07:49:36.541116 12934 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 38 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:36.541244 12538 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:36.541308 12934 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 7 FOLLOWER]: Advancing to term 8
I20250905 07:49:36.541481 12538 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 7.38, Last appended by leader: 38, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:36.541744 12538 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 7 FOLLOWER]: Advancing to term 8
I20250905 07:49:36.543224 12934 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 8.
I20250905 07:49:36.543632 12432 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:36.543731 12538 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 8.
I20250905 07:49:36.543861 13461 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 FOLLOWER]: Leader election won for term 8
I20250905 07:49:36.544519 13461 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:36.544749 13461 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 7.38, Last appended by leader: 38, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:36.547324 12359 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 7 to 8, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 8 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:37.019863 12538 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 8 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 38. Preceding OpId from leader: term: 8 index: 39. (index mismatch)
I20250905 07:49:37.020371 13461 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20250905 07:49:37.024413 12934 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 8 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 38. Preceding OpId from leader: term: 8 index: 39. (index mismatch)
I20250905 07:49:37.025218 13482 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20250905 07:49:38.116909 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:38.119581 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:38.119786 12465 raft_consensus.cc:604] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Received request to transfer leadership to TS 0e02ae2f2eb74d7ab0440a758107e204
I20250905 07:49:38.120229 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:38.120427 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:38.574869 13496 raft_consensus.cc:991] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:38.575078 13496 raft_consensus.cc:1079] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Signalling peer 0e02ae2f2eb74d7ab0440a758107e204 to start an election
I20250905 07:49:38.575783 12858 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7"
dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
from {username='slave'} at 127.0.0.1:48294
I20250905 07:49:38.576023 12858 raft_consensus.cc:491] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:38.576149 12858 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Advancing to term 6
I20250905 07:49:38.578186 12858 raft_consensus.cc:513] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:38.578817 12858 leader_election.cc:290] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 6 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:38.579282 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 6 candidate_status { last_received { term: 5 index: 39 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:38.579324 12934 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69a843a1d994abea9c6ec7321f6c3d7" candidate_uuid: "0e02ae2f2eb74d7ab0440a758107e204" candidate_term: 6 candidate_status { last_received { term: 5 index: 39 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:38.579491 12465 raft_consensus.cc:3053] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Stepping down as leader of term 5
I20250905 07:49:38.579535 12934 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 5 FOLLOWER]: Advancing to term 6
I20250905 07:49:38.579587 12465 raft_consensus.cc:738] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:38.579866 12465 consensus_queue.cc:260] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 5.39, Last appended by leader: 39, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:38.580176 12465 raft_consensus.cc:3058] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Advancing to term 6
I20250905 07:49:38.581661 12934 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 6.
I20250905 07:49:38.581923 12465 raft_consensus.cc:2466] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02ae2f2eb74d7ab0440a758107e204 in term 6.
I20250905 07:49:38.582259 12824 leader_election.cc:304] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02ae2f2eb74d7ab0440a758107e204, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:38.582535 13433 raft_consensus.cc:2802] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 6 FOLLOWER]: Leader election won for term 6
I20250905 07:49:38.582887 13433 raft_consensus.cc:695] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 6 LEADER]: Becoming Leader. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Running, Role: LEADER
I20250905 07:49:38.583173 13433 consensus_queue.cc:237] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 5.39, Last appended by leader: 39, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:38.585675 12359 catalog_manager.cc:5582] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 reported cstate change: term changed from 5 to 6, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68). New cstate: current_term: 6 leader_uuid: "0e02ae2f2eb74d7ab0440a758107e204" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:39.130169 12934 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 5 index: 39. Preceding OpId from leader: term: 6 index: 40. (index mismatch)
I20250905 07:49:39.130708 13433 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20250905 07:49:39.134771 12465 raft_consensus.cc:1273] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 FOLLOWER]: Refusing update from remote peer 0e02ae2f2eb74d7ab0440a758107e204: Log matching property violated. Preceding OpId in replica: term: 5 index: 39. Preceding OpId from leader: term: 6 index: 40. (index mismatch)
I20250905 07:49:39.136643 13508 consensus_queue.cc:1035] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20250905 07:49:40.120843 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:40.123672 12858 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:50800
I20250905 07:49:40.123896 12858 raft_consensus.cc:604] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:40.124408 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:40.124624 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:40.574698 13504 raft_consensus.cc:991] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:40.574895 13504 raft_consensus.cc:1079] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:40.575512 12465 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59228
I20250905 07:49:40.575703 12465 raft_consensus.cc:491] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:40.575820 12465 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:40.577805 12465 raft_consensus.cc:513] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:40.578476 12465 leader_election.cc:290] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 5 election: Requested vote from peers 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511), 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:40.578826 12612 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:40.578925 12858 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a62c7bcdb1a497898d2078a2f891cff" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: true dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
I20250905 07:49:40.579032 12612 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:40.579108 12858 raft_consensus.cc:3053] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Stepping down as leader of term 4
I20250905 07:49:40.579226 12858 raft_consensus.cc:738] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e02ae2f2eb74d7ab0440a758107e204, State: Running, Role: LEADER
I20250905 07:49:40.579520 12858 consensus_queue.cc:260] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:40.579843 12858 raft_consensus.cc:3058] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Advancing to term 5
I20250905 07:49:40.581352 12612 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 5.
I20250905 07:49:40.581744 12433 leader_election.cc:304] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, 5531c3eeac82412fb18050ea7148b03e; no voters:
I20250905 07:49:40.581853 12858 raft_consensus.cc:2466] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 5.
I20250905 07:49:40.582022 13496 raft_consensus.cc:2802] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Leader election won for term 5
I20250905 07:49:40.582165 13496 raft_consensus.cc:695] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:40.582440 13496 consensus_queue.cc:237] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:40.584846 12357 catalog_manager.cc:5582] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 4 to 5, leader changed from 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 5 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:40.987731 12612 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20250905 07:49:40.988286 13496 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:40.991899 12858 raft_consensus.cc:1273] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20250905 07:49:40.992589 13496 consensus_queue.cc:1035] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20250905 07:49:42.125016 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:42.127810 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:42.128012 12465 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:42.128517 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:42.128736 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:42.282562 13496 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:42.282718 13496 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:42.283382 12538 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58386
I20250905 07:49:42.283622 12538 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:42.283733 12538 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 8 FOLLOWER]: Advancing to term 9
I20250905 07:49:42.285633 12538 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:42.286298 12538 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 9 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:42.286681 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 9 candidate_status { last_received { term: 8 index: 39 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:42.286768 12934 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 9 candidate_status { last_received { term: 8 index: 39 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:42.286865 12465 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 LEADER]: Stepping down as leader of term 8
I20250905 07:49:42.286953 12934 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 8 FOLLOWER]: Advancing to term 9
I20250905 07:49:42.286978 12465 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:42.287243 12465 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 8.39, Last appended by leader: 39, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:42.287573 12465 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 8 FOLLOWER]: Advancing to term 9
I20250905 07:49:42.288873 12934 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 9.
I20250905 07:49:42.289351 12507 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:42.289592 12465 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 9.
I20250905 07:49:42.289604 13397 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 FOLLOWER]: Leader election won for term 9
I20250905 07:49:42.290133 13397 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:42.290447 13397 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 8.39, Last appended by leader: 39, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:42.292994 12359 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 8 to 9, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 9 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:42.672327 12465 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 9 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 8 index: 39. Preceding OpId from leader: term: 9 index: 40. (index mismatch)
I20250905 07:49:42.672863 13397 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20250905 07:49:42.677220 12934 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 9 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 8 index: 39. Preceding OpId from leader: term: 9 index: 40. (index mismatch)
I20250905 07:49:42.677742 13527 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20250905 07:49:44.129125 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:44.131655 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:44.131829 12538 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:44.132185 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:44.132352 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:44.141993 13541 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:44.142153 13541 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:44.142663 12465 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59180
I20250905 07:49:44.142820 12465 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:44.142925 12465 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 9 FOLLOWER]: Advancing to term 10
I20250905 07:49:44.144758 12465 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:44.145428 12465 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 10 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:44.145764 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:44.145828 12934 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:44.145927 12934 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 9 FOLLOWER]: Advancing to term 10
I20250905 07:49:44.145927 12538 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 LEADER]: Stepping down as leader of term 9
I20250905 07:49:44.146042 12538 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:44.146263 12538 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 9.40, Last appended by leader: 40, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:44.146564 12538 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 9 FOLLOWER]: Advancing to term 10
I20250905 07:49:44.148586 12934 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 10.
I20250905 07:49:44.148972 12432 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df0c6def6514c2a859a74187f5d2d8e, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:44.149019 12538 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 10.
I20250905 07:49:44.149253 13496 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 FOLLOWER]: Leader election won for term 10
I20250905 07:49:44.149588 13496 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:44.149783 13496 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 9.40, Last appended by leader: 40, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:44.151988 12357 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 9 to 10, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 10 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:44.559217 12934 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 10 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 40. Preceding OpId from leader: term: 10 index: 41. (index mismatch)
I20250905 07:49:44.559798 13496 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20250905 07:49:44.563416 12538 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 10 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 40. Preceding OpId from leader: term: 10 index: 41. (index mismatch)
I20250905 07:49:44.563946 13496 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20250905 07:49:46.132741 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:46.135504 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:46.135717 12465 raft_consensus.cc:604] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:46.136161 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:46.136369 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:46.467902 13557 raft_consensus.cc:991] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:46.468098 13557 raft_consensus.cc:1079] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:46.468781 12538 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "89d6356d89d8404e94394907a2222a62"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58386
I20250905 07:49:46.468966 12538 raft_consensus.cc:491] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:46.469092 12538 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 10 FOLLOWER]: Advancing to term 11
I20250905 07:49:46.471027 12538 raft_consensus.cc:513] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:46.471577 12538 leader_election.cc:290] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 11 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), af2950144b8c46438eec51827c06eb94 (127.8.229.69:42245)
I20250905 07:49:46.471995 12934 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 11 candidate_status { last_received { term: 10 index: 41 } } ignore_live_leader: true dest_uuid: "af2950144b8c46438eec51827c06eb94"
I20250905 07:49:46.472141 12934 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 10 FOLLOWER]: Advancing to term 11
I20250905 07:49:46.472076 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "89d6356d89d8404e94394907a2222a62" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 11 candidate_status { last_received { term: 10 index: 41 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:46.472262 12465 raft_consensus.cc:3053] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 LEADER]: Stepping down as leader of term 10
I20250905 07:49:46.472347 12465 raft_consensus.cc:738] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:46.472636 12465 consensus_queue.cc:260] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 10.41, Last appended by leader: 41, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:46.473006 12465 raft_consensus.cc:3058] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 10 FOLLOWER]: Advancing to term 11
I20250905 07:49:46.474020 12934 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 11.
I20250905 07:49:46.474457 12507 leader_election.cc:304] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, af2950144b8c46438eec51827c06eb94; no voters:
I20250905 07:49:46.474668 13541 raft_consensus.cc:2802] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 11 FOLLOWER]: Leader election won for term 11
I20250905 07:49:46.475075 13541 raft_consensus.cc:695] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 11 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:46.475327 13541 consensus_queue.cc:237] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 10.41, Last appended by leader: 41, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } }
I20250905 07:49:46.477226 12465 raft_consensus.cc:2466] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 11.
I20250905 07:49:46.477983 12359 catalog_manager.cc:5582] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 10 to 11, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 11 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:46.950130 12465 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 11 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 10 index: 41. Preceding OpId from leader: term: 11 index: 42. (index mismatch)
I20250905 07:49:46.950690 13541 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20250905 07:49:46.959072 12934 raft_consensus.cc:1273] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 11 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 10 index: 41. Preceding OpId from leader: term: 11 index: 42. (index mismatch)
I20250905 07:49:46.959563 13558 consensus_queue.cc:1035] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2950144b8c46438eec51827c06eb94" member_type: VOTER last_known_addr { host: "127.8.229.69" port: 42245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20250905 07:49:48.136689 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:48.139570 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:48.139827 12538 raft_consensus.cc:604] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Received request to transfer leadership to TS 2df0c6def6514c2a859a74187f5d2d8e
I20250905 07:49:48.140328 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:48.140502 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:48.209525 13558 raft_consensus.cc:991] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:48.209674 13558 raft_consensus.cc:1079] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Signalling peer 2df0c6def6514c2a859a74187f5d2d8e to start an election
I20250905 07:49:48.210213 12465 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
from {username='slave'} at 127.0.0.1:59180
I20250905 07:49:48.210390 12465 raft_consensus.cc:491] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:48.210461 12465 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:48.212417 12465 raft_consensus.cc:513] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:48.212988 12465 leader_election.cc:290] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 3 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:48.213459 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:48.213603 12858 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "2df0c6def6514c2a859a74187f5d2d8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
I20250905 07:49:48.213631 12538 raft_consensus.cc:3053] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:49:48.213721 12538 raft_consensus.cc:738] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:48.213742 12858 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:48.213992 12538 consensus_queue.cc:260] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:48.214299 12538 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:48.216435 12858 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 3.
I20250905 07:49:48.216863 12430 leader_election.cc:304] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02ae2f2eb74d7ab0440a758107e204, 2df0c6def6514c2a859a74187f5d2d8e; no voters:
I20250905 07:49:48.217018 12538 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df0c6def6514c2a859a74187f5d2d8e in term 3.
I20250905 07:49:48.217191 13557 raft_consensus.cc:2802] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:49:48.217636 13557 raft_consensus.cc:695] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Becoming Leader. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:48.217890 13557 consensus_queue.cc:237] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:48.220070 12357 catalog_manager.cc:5582] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e reported cstate change: term changed from 2 to 3, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65). New cstate: current_term: 3 leader_uuid: "2df0c6def6514c2a859a74187f5d2d8e" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:48.719339 12538 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20250905 07:49:48.719890 13557 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:48.723989 12858 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Refusing update from remote peer 2df0c6def6514c2a859a74187f5d2d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20250905 07:49:48.724684 13557 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20250905 07:49:50.140875 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:50.144107 12465 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28"
dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
mode: GRACEFUL
new_leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:59212
I20250905 07:49:50.144333 12465 raft_consensus.cc:604] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Received request to transfer leadership to TS 069247fbe8b34bbd81d3549f0fc0b93a
I20250905 07:49:50.144824 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:50.145004 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:50.259758 13585 raft_consensus.cc:991] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e: : Instructing follower 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:50.259966 13585 raft_consensus.cc:1079] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Signalling peer 069247fbe8b34bbd81d3549f0fc0b93a to start an election
I20250905 07:49:50.260537 12538 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
from {username='slave'} at 127.0.0.1:58386
I20250905 07:49:50.260704 12538 raft_consensus.cc:491] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:50.260788 12538 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:50.262674 12538 raft_consensus.cc:513] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:50.263319 12538 leader_election.cc:290] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 4 election: Requested vote from peers 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247), 0e02ae2f2eb74d7ab0440a758107e204 (127.8.229.68:33403)
I20250905 07:49:50.263715 12858 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: true dest_uuid: "0e02ae2f2eb74d7ab0440a758107e204"
I20250905 07:49:50.263743 12465 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5209f5aac7a24f62a3b520b55ae51a28" candidate_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: true dest_uuid: "2df0c6def6514c2a859a74187f5d2d8e"
I20250905 07:49:50.263928 12858 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:50.264016 12465 raft_consensus.cc:3053] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Stepping down as leader of term 3
I20250905 07:49:50.264111 12465 raft_consensus.cc:738] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2df0c6def6514c2a859a74187f5d2d8e, State: Running, Role: LEADER
I20250905 07:49:50.264369 12465 consensus_queue.cc:260] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:50.264629 12465 raft_consensus.cc:3058] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 3 FOLLOWER]: Advancing to term 4
I20250905 07:49:50.265931 12858 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 4.
I20250905 07:49:50.266294 12505 leader_election.cc:304] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 069247fbe8b34bbd81d3549f0fc0b93a, 0e02ae2f2eb74d7ab0440a758107e204; no voters:
I20250905 07:49:50.266543 13558 raft_consensus.cc:2802] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Leader election won for term 4
I20250905 07:49:50.266665 12465 raft_consensus.cc:2466] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 069247fbe8b34bbd81d3549f0fc0b93a in term 4.
I20250905 07:49:50.266939 13558 raft_consensus.cc:695] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 LEADER]: Becoming Leader. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:50.267210 13558 consensus_queue.cc:237] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } }
I20250905 07:49:50.269728 12357 catalog_manager.cc:5582] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a reported cstate change: term changed from 3 to 4, leader changed from 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65) to 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66). New cstate: current_term: 4 leader_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:50.687008 12465 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20250905 07:49:50.687522 13558 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df0c6def6514c2a859a74187f5d2d8e" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:50.691620 12858 raft_consensus.cc:1273] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Refusing update from remote peer 069247fbe8b34bbd81d3549f0fc0b93a: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20250905 07:49:50.692188 13558 consensus_queue.cc:1035] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e02ae2f2eb74d7ab0440a758107e204" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20250905 07:49:52.145421 12418 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:52.148160 12538 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed"
dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
mode: GRACEFUL
new_leader_uuid: "99e007e4f8594df8b4e2aea848ed050c"
from {username='slave'} at 127.0.0.1:58400
I20250905 07:49:52.148386 12538 raft_consensus.cc:604] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Received request to transfer leadership to TS 99e007e4f8594df8b4e2aea848ed050c
I20250905 07:49:52.148895 12418 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:52.149073 12418 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:52.280975 13598 raft_consensus.cc:991] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: : Instructing follower 99e007e4f8594df8b4e2aea848ed050c to start an election
I20250905 07:49:52.281180 13598 raft_consensus.cc:1079] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Signalling peer 99e007e4f8594df8b4e2aea848ed050c to start an election
I20250905 07:49:52.281762 13007 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed"
dest_uuid: "99e007e4f8594df8b4e2aea848ed050c"
from {username='slave'} at 127.0.0.1:52790
I20250905 07:49:52.281956 13007 raft_consensus.cc:491] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:52.282073 13007 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:52.284396 13007 raft_consensus.cc:513] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:52.285118 13007 leader_election.cc:290] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [CANDIDATE]: Term 3 election: Requested vote from peers 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66:36723), 5531c3eeac82412fb18050ea7148b03e (127.8.229.67:34511)
I20250905 07:49:52.285558 12538 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "99e007e4f8594df8b4e2aea848ed050c" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "069247fbe8b34bbd81d3549f0fc0b93a"
I20250905 07:49:52.285558 12612 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "655c91ccdcf4422a8795cecf130a56ed" candidate_uuid: "99e007e4f8594df8b4e2aea848ed050c" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "5531c3eeac82412fb18050ea7148b03e"
I20250905 07:49:52.285804 12612 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:52.285804 12538 raft_consensus.cc:3053] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:49:52.285951 12538 raft_consensus.cc:738] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 069247fbe8b34bbd81d3549f0fc0b93a, State: Running, Role: LEADER
I20250905 07:49:52.286221 12538 consensus_queue.cc:260] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:52.286576 12538 raft_consensus.cc:3058] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:49:52.288731 12612 raft_consensus.cc:2466] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e007e4f8594df8b4e2aea848ed050c in term 3.
I20250905 07:49:52.289209 12538 raft_consensus.cc:2466] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e007e4f8594df8b4e2aea848ed050c in term 3.
I20250905 07:49:52.289242 12974 leader_election.cc:304] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5531c3eeac82412fb18050ea7148b03e, 99e007e4f8594df8b4e2aea848ed050c; no voters:
I20250905 07:49:52.289525 13495 raft_consensus.cc:2802] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:49:52.289930 13495 raft_consensus.cc:695] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 3 LEADER]: Becoming Leader. State: Replica: 99e007e4f8594df8b4e2aea848ed050c, State: Running, Role: LEADER
I20250905 07:49:52.290197 13495 consensus_queue.cc:237] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } }
I20250905 07:49:52.292454 12359 catalog_manager.cc:5582] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c reported cstate change: term changed from 2 to 3, leader changed from 069247fbe8b34bbd81d3549f0fc0b93a (127.8.229.66) to 99e007e4f8594df8b4e2aea848ed050c (127.8.229.70). New cstate: current_term: 3 leader_uuid: "99e007e4f8594df8b4e2aea848ed050c" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "069247fbe8b34bbd81d3549f0fc0b93a" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5531c3eeac82412fb18050ea7148b03e" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e007e4f8594df8b4e2aea848ed050c" member_type: VOTER last_known_addr { host: "127.8.229.70" port: 44069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_rebalancer-test.cc:195: Failure
Expected equality of these values:
0
NumMovesScheduled(leader_idx, BalanceThreadType::LEADER_REBALANCE)
Which is: 1
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_rebalancer-test.cc:575: Failure
Expected: CheckNoLeaderMovesScheduled() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20250905 07:49:52.437454 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
W20250905 07:49:52.439044 12823 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20250905 07:49:52.442642 12823 consensus_peers.cc:489] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 -> Peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): Couldn't send request to peer 2df0c6def6514c2a859a74187f5d2d8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:52.453518 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
W20250905 07:49:52.454152 12506 consensus_peers.cc:489] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a -> Peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): Couldn't send request to peer 2df0c6def6514c2a859a74187f5d2d8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:52.454216 9109 tablet_replica.cc:331] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:52.454440 9109 raft_consensus.cc:2241] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.454645 9109 raft_consensus.cc:2270] T 5209f5aac7a24f62a3b520b55ae51a28 P 2df0c6def6514c2a859a74187f5d2d8e [term 4 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.455464 9109 tablet_replica.cc:331] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:52.455654 9109 raft_consensus.cc:2241] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 5 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.455978 9109 raft_consensus.cc:2270] T 3a62c7bcdb1a497898d2078a2f891cff P 2df0c6def6514c2a859a74187f5d2d8e [term 5 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.456532 9109 tablet_replica.cc:331] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:52.456660 9109 raft_consensus.cc:2241] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 11 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.456815 9109 raft_consensus.cc:2270] T 89d6356d89d8404e94394907a2222a62 P 2df0c6def6514c2a859a74187f5d2d8e [term 11 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.457264 9109 tablet_replica.cc:331] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:52.457417 9109 raft_consensus.cc:2241] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.457548 9109 raft_consensus.cc:2270] T c69a843a1d994abea9c6ec7321f6c3d7 P 2df0c6def6514c2a859a74187f5d2d8e [term 6 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.457981 9109 tablet_replica.cc:331] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:52.458127 9109 raft_consensus.cc:2241] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.458434 9109 raft_consensus.cc:2270] T b380bffe76bd498e8d98e64dae8728ab P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.458866 9109 tablet_replica.cc:331] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e: stopping tablet replica
I20250905 07:49:52.459007 9109 raft_consensus.cc:2241] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.459267 9109 raft_consensus.cc:2270] T f919efd17c674fe799365a7d79d969d1 P 2df0c6def6514c2a859a74187f5d2d8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.491564 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:49:52.496754 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:49:52.506450 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:52.506750 9109 tablet_replica.cc:331] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:52.506932 9109 raft_consensus.cc:2241] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.507259 9109 raft_consensus.cc:2270] T 5209f5aac7a24f62a3b520b55ae51a28 P 069247fbe8b34bbd81d3549f0fc0b93a [term 4 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.507921 9109 tablet_replica.cc:331] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:52.508070 9109 raft_consensus.cc:2241] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 11 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.508373 9109 raft_consensus.cc:2270] T 89d6356d89d8404e94394907a2222a62 P 069247fbe8b34bbd81d3549f0fc0b93a [term 11 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.508904 9109 tablet_replica.cc:331] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:52.509050 9109 raft_consensus.cc:2241] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.509169 9109 raft_consensus.cc:2270] T 3f902eed21ca4a1abaa2335e791c9fd5 P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.509621 9109 tablet_replica.cc:331] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:52.509742 9109 raft_consensus.cc:2241] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.509927 9109 raft_consensus.cc:2270] T 39d264067d4447c592529d8bf6e39eac P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.510342 9109 tablet_replica.cc:331] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:52.510464 9109 raft_consensus.cc:2241] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.510587 9109 raft_consensus.cc:2270] T b380bffe76bd498e8d98e64dae8728ab P 069247fbe8b34bbd81d3549f0fc0b93a [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.510977 9109 tablet_replica.cc:331] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a: stopping tablet replica
I20250905 07:49:52.511096 9109 raft_consensus.cc:2241] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.511219 9109 raft_consensus.cc:2270] T 655c91ccdcf4422a8795cecf130a56ed P 069247fbe8b34bbd81d3549f0fc0b93a [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.528952 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:49:52.534044 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:49:52.557544 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:52.557833 9109 tablet_replica.cc:331] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:52.558024 9109 raft_consensus.cc:2241] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.558336 9109 raft_consensus.cc:2270] T 3467a871458e47d488238ece84ae086b P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.558965 9109 tablet_replica.cc:331] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:52.559115 9109 raft_consensus.cc:2241] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 3 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.559360 9109 raft_consensus.cc:2270] T 559347a6dcac472c9b09b80a16527175 P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.559842 9109 tablet_replica.cc:331] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:52.559973 9109 raft_consensus.cc:2241] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.560181 9109 raft_consensus.cc:2270] T 3a62c7bcdb1a497898d2078a2f891cff P 5531c3eeac82412fb18050ea7148b03e [term 5 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.560597 9109 tablet_replica.cc:331] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:52.560726 9109 raft_consensus.cc:2241] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.560909 9109 raft_consensus.cc:2270] T 3f902eed21ca4a1abaa2335e791c9fd5 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.561305 9109 tablet_replica.cc:331] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:52.561427 9109 raft_consensus.cc:2241] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.561620 9109 raft_consensus.cc:2270] T 655c91ccdcf4422a8795cecf130a56ed P 5531c3eeac82412fb18050ea7148b03e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.562052 9109 tablet_replica.cc:331] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e: stopping tablet replica
I20250905 07:49:52.562155 9109 raft_consensus.cc:2241] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.562320 9109 raft_consensus.cc:2270] T 0e94f73c59d74a2a862c445f8f70e618 P 5531c3eeac82412fb18050ea7148b03e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.578148 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:49:52.583225 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:49:52.592662 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:52.592944 9109 tablet_replica.cc:331] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204: stopping tablet replica
I20250905 07:49:52.593130 9109 raft_consensus.cc:2241] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 6 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.593452 9109 raft_consensus.cc:2270] T c69a843a1d994abea9c6ec7321f6c3d7 P 0e02ae2f2eb74d7ab0440a758107e204 [term 6 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.594028 9109 tablet_replica.cc:331] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204: stopping tablet replica
I20250905 07:49:52.594172 9109 raft_consensus.cc:2241] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.594319 9109 raft_consensus.cc:2270] T 5209f5aac7a24f62a3b520b55ae51a28 P 0e02ae2f2eb74d7ab0440a758107e204 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.594749 9109 tablet_replica.cc:331] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204: stopping tablet replica
I20250905 07:49:52.594882 9109 raft_consensus.cc:2241] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.595008 9109 raft_consensus.cc:2270] T 3a62c7bcdb1a497898d2078a2f891cff P 0e02ae2f2eb74d7ab0440a758107e204 [term 5 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.595415 9109 tablet_replica.cc:331] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204: stopping tablet replica
I20250905 07:49:52.595605 9109 raft_consensus.cc:2241] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.595781 9109 raft_consensus.cc:2270] T 39d264067d4447c592529d8bf6e39eac P 0e02ae2f2eb74d7ab0440a758107e204 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.596189 9109 tablet_replica.cc:331] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204: stopping tablet replica
I20250905 07:49:52.596309 9109 raft_consensus.cc:2241] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.596534 9109 raft_consensus.cc:2270] T 0e94f73c59d74a2a862c445f8f70e618 P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.596925 9109 tablet_replica.cc:331] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204: stopping tablet replica
I20250905 07:49:52.597043 9109 raft_consensus.cc:2241] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.597234 9109 raft_consensus.cc:2270] T 3467a871458e47d488238ece84ae086b P 0e02ae2f2eb74d7ab0440a758107e204 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.612010 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:49:52.615888 9109 tablet_server.cc:178] TabletServer@127.8.229.69:0 shutting down...
I20250905 07:49:52.626117 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:52.626379 9109 tablet_replica.cc:331] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94: stopping tablet replica
I20250905 07:49:52.626569 9109 raft_consensus.cc:2241] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.626837 9109 raft_consensus.cc:2270] T c69a843a1d994abea9c6ec7321f6c3d7 P af2950144b8c46438eec51827c06eb94 [term 6 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.627460 9109 tablet_replica.cc:331] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94: stopping tablet replica
I20250905 07:49:52.627621 9109 raft_consensus.cc:2241] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.627818 9109 raft_consensus.cc:2270] T f919efd17c674fe799365a7d79d969d1 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.628286 9109 tablet_replica.cc:331] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94: stopping tablet replica
I20250905 07:49:52.628413 9109 raft_consensus.cc:2241] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 11 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.628551 9109 raft_consensus.cc:2270] T 89d6356d89d8404e94394907a2222a62 P af2950144b8c46438eec51827c06eb94 [term 11 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.628953 9109 tablet_replica.cc:331] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94: stopping tablet replica
I20250905 07:49:52.629087 9109 raft_consensus.cc:2241] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 3 LEADER]: Raft consensus shutting down.
W20250905 07:49:52.630977 12972 consensus_peers.cc:489] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c -> Peer 2df0c6def6514c2a859a74187f5d2d8e (127.8.229.65:35247): Couldn't send request to peer 2df0c6def6514c2a859a74187f5d2d8e. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:35247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:52.646880 9109 raft_consensus.cc:2270] T 39d264067d4447c592529d8bf6e39eac P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.647634 9109 tablet_replica.cc:331] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94: stopping tablet replica
I20250905 07:49:52.647783 9109 raft_consensus.cc:2241] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.648063 9109 raft_consensus.cc:2270] T 3f902eed21ca4a1abaa2335e791c9fd5 P af2950144b8c46438eec51827c06eb94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.648526 9109 tablet_replica.cc:331] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94: stopping tablet replica
I20250905 07:49:52.648620 9109 raft_consensus.cc:2241] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.648900 9109 raft_consensus.cc:2270] T 559347a6dcac472c9b09b80a16527175 P af2950144b8c46438eec51827c06eb94 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.663311 9109 tablet_server.cc:195] TabletServer@127.8.229.69:0 shutdown complete.
I20250905 07:49:52.667225 9109 tablet_server.cc:178] TabletServer@127.8.229.70:0 shutting down...
I20250905 07:49:52.677070 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:49:52.677357 9109 tablet_replica.cc:331] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c: stopping tablet replica
I20250905 07:49:52.677547 9109 raft_consensus.cc:2241] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.677752 9109 raft_consensus.cc:2270] T 559347a6dcac472c9b09b80a16527175 P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.678308 9109 tablet_replica.cc:331] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c: stopping tablet replica
I20250905 07:49:52.678423 9109 raft_consensus.cc:2241] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.678597 9109 raft_consensus.cc:2270] T f919efd17c674fe799365a7d79d969d1 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.678974 9109 tablet_replica.cc:331] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c: stopping tablet replica
I20250905 07:49:52.679078 9109 raft_consensus.cc:2241] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.679219 9109 raft_consensus.cc:2270] T 0e94f73c59d74a2a862c445f8f70e618 P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.679622 9109 tablet_replica.cc:331] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c: stopping tablet replica
I20250905 07:49:52.679718 9109 raft_consensus.cc:2241] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:52.679854 9109 raft_consensus.cc:2270] T 3467a871458e47d488238ece84ae086b P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.680222 9109 tablet_replica.cc:331] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: stopping tablet replica
I20250905 07:49:52.680316 9109 raft_consensus.cc:2241] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 3 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.680508 9109 pending_rounds.cc:70] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Trying to abort 1 pending ops.
I20250905 07:49:52.680544 9109 pending_rounds.cc:77] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c: Aborting op as it isn't in flight: id { term: 3 index: 34 } timestamp: 7196911994022785024 op_type: NO_OP noop_request { }
I20250905 07:49:52.680634 9109 raft_consensus.cc:2887] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:49:52.680691 9109 raft_consensus.cc:2270] T 655c91ccdcf4422a8795cecf130a56ed P 99e007e4f8594df8b4e2aea848ed050c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.681221 9109 tablet_replica.cc:331] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c: stopping tablet replica
I20250905 07:49:52.681314 9109 raft_consensus.cc:2241] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.681522 9109 raft_consensus.cc:2270] T b380bffe76bd498e8d98e64dae8728ab P 99e007e4f8594df8b4e2aea848ed050c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.696245 9109 tablet_server.cc:195] TabletServer@127.8.229.70:0 shutdown complete.
I20250905 07:49:52.700026 9109 master.cc:561] Master@127.8.229.126:45145 shutting down...
I20250905 07:49:52.708389 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:49:52.708604 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:52.708722 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 0080ac993145405cb9ce0d80deff9150: stopping tablet replica
I20250905 07:49:52.712389 9109 master.cc:583] Master@127.8.229.126:45145 shutdown complete.
I20250905 07:49:52.735163 9109 test_util.cc:183] -----------------------------------------------
I20250905 07:49:52.735245 9109 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1757058461885731-9109-0
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer (37140 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingUnstableCluster
I20250905 07:49:52.737100 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:33945
I20250905 07:49:52.737610 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:52.740185 13610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:52.740185 13613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:52.740439 9109 server_base.cc:1047] running on GCE node
W20250905 07:49:52.740454 13611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:52.740913 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:52.740984 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:52.741008 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058592741005 us; error 0 us; skew 500 ppm
I20250905 07:49:52.741137 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:52.742233 9109 webserver.cc:480] Webserver started at http://127.8.229.126:33801/ using document root <none> and password file <none>
I20250905 07:49:52.742442 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:52.742506 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:52.742632 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:52.743239 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "8f55ada7d81b46d497a0a9584354caa7"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.745667 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20250905 07:49:52.747124 13618 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.747471 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:49:52.747567 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "8f55ada7d81b46d497a0a9584354caa7"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.747675 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:52.769119 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:52.769667 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:52.779726 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:33945
I20250905 07:49:52.779749 13669 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:33945 every 8 connection(s)
I20250905 07:49:52.780931 13670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:52.784785 13670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7: Bootstrap starting.
I20250905 07:49:52.786008 13670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:52.787462 13670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7: No bootstrap required, opened a new log
I20250905 07:49:52.787972 13670 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f55ada7d81b46d497a0a9584354caa7" member_type: VOTER }
I20250905 07:49:52.788121 13670 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:52.788164 13670 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f55ada7d81b46d497a0a9584354caa7, State: Initialized, Role: FOLLOWER
I20250905 07:49:52.788304 13670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f55ada7d81b46d497a0a9584354caa7" member_type: VOTER }
I20250905 07:49:52.788408 13670 raft_consensus.cc:397] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:49:52.788447 13670 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:49:52.788496 13670 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:52.790071 13670 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f55ada7d81b46d497a0a9584354caa7" member_type: VOTER }
I20250905 07:49:52.790223 13670 leader_election.cc:304] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f55ada7d81b46d497a0a9584354caa7; no voters:
I20250905 07:49:52.790532 13670 leader_election.cc:290] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:49:52.790640 13673 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:52.791043 13673 raft_consensus.cc:695] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 1 LEADER]: Becoming Leader. State: Replica: 8f55ada7d81b46d497a0a9584354caa7, State: Running, Role: LEADER
I20250905 07:49:52.791237 13673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f55ada7d81b46d497a0a9584354caa7" member_type: VOTER }
I20250905 07:49:52.791290 13670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:49:52.792021 13675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f55ada7d81b46d497a0a9584354caa7. Latest consensus state: current_term: 1 leader_uuid: "8f55ada7d81b46d497a0a9584354caa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f55ada7d81b46d497a0a9584354caa7" member_type: VOTER } }
I20250905 07:49:52.792192 13675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [sys.catalog]: This master's current role is: LEADER
I20250905 07:49:52.792461 13674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f55ada7d81b46d497a0a9584354caa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f55ada7d81b46d497a0a9584354caa7" member_type: VOTER } }
I20250905 07:49:52.792641 13674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [sys.catalog]: This master's current role is: LEADER
I20250905 07:49:52.793855 13678 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:49:52.795794 13678 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:49:52.796607 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:49:52.798688 13678 catalog_manager.cc:1349] Generated new cluster ID: f0ba0943775041c6a950c1168589ff20
I20250905 07:49:52.798754 13678 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:49:52.824739 13678 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:49:52.825500 13678 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:49:52.833232 13678 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7: Generated new TSK 0
I20250905 07:49:52.833441 13678 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:49:52.861596 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:52.864089 13691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:52.864405 13692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:52.864492 13694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:52.865304 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:52.865532 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:52.865581 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:52.865602 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058592865599 us; error 0 us; skew 500 ppm
I20250905 07:49:52.865698 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:52.866796 9109 webserver.cc:480] Webserver started at http://127.8.229.65:39023/ using document root <none> and password file <none>
I20250905 07:49:52.866981 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:52.867034 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:52.867130 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:52.867774 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "ca0a744b5f0c46758a3e58e324f37373"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.870132 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:52.871547 13699 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.871930 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:52.872017 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "ca0a744b5f0c46758a3e58e324f37373"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.872123 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:52.884112 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:52.884516 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:52.884979 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:52.885748 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:52.885799 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.885852 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:52.885881 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.899101 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:33645
I20250905 07:49:52.899129 13761 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:33645 every 8 connection(s)
I20250905 07:49:52.900672 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:52.903321 13766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:52.903517 13767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:52.903992 13769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:52.904819 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:52.905041 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:52.905093 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:52.905122 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058592905118 us; error 0 us; skew 500 ppm
I20250905 07:49:52.905246 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:52.907308 9109 webserver.cc:480] Webserver started at http://127.8.229.66:44515/ using document root <none> and password file <none>
I20250905 07:49:52.907545 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:52.907613 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:52.907728 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:52.907927 13762 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33945
I20250905 07:49:52.908067 13762 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:52.908454 13762 heartbeater.cc:507] Master 127.8.229.126:33945 requested a full tablet report, sending...
I20250905 07:49:52.908454 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "d75f639992a44f4f8bfeb0a77ba4b469"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.909286 13635 ts_manager.cc:194] Registered new tserver with Master: ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645)
I20250905 07:49:52.910262 13635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41670
I20250905 07:49:52.910996 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:52.912482 13774 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.912750 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:52.912834 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "d75f639992a44f4f8bfeb0a77ba4b469"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.912925 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:52.931110 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:52.931643 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:52.932195 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:52.932968 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:52.933024 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.933105 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:52.933145 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.946060 13836 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:40761 every 8 connection(s)
I20250905 07:49:52.946039 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:40761
I20250905 07:49:52.947399 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:52.950067 13840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:52.950629 13841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:52.951751 9109 server_base.cc:1047] running on GCE node
W20250905 07:49:52.953186 13843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:52.953498 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:52.953552 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:52.953575 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058592953572 us; error 0 us; skew 500 ppm
I20250905 07:49:52.953699 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:52.954409 13837 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33945
I20250905 07:49:52.954535 13837 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:52.954936 13837 heartbeater.cc:507] Master 127.8.229.126:33945 requested a full tablet report, sending...
I20250905 07:49:52.954962 9109 webserver.cc:480] Webserver started at http://127.8.229.67:43593/ using document root <none> and password file <none>
I20250905 07:49:52.955195 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:52.955282 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:52.955408 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:52.955801 13635 ts_manager.cc:194] Registered new tserver with Master: d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761)
I20250905 07:49:52.956063 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.956547 13635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41682
I20250905 07:49:52.958736 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:49:52.960268 13848 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.960572 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:49:52.960664 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b"
format_stamp: "Formatted at 2025-09-05 07:49:52 on dist-test-slave-pgzt"
I20250905 07:49:52.960764 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:52.977072 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:52.977584 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:52.978163 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:52.979005 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:52.979063 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.979135 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:52.979172 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:52.991950 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:42045
I20250905 07:49:52.991976 13910 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:42045 every 8 connection(s)
I20250905 07:49:52.998149 13911 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33945
I20250905 07:49:52.998279 13911 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:52.998610 13911 heartbeater.cc:507] Master 127.8.229.126:33945 requested a full tablet report, sending...
I20250905 07:49:52.999392 13635 ts_manager.cc:194] Registered new tserver with Master: c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:52.999665 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006832647s
I20250905 07:49:53.000382 13635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41686
I20250905 07:49:53.911645 13762 heartbeater.cc:499] Master 127.8.229.126:33945 was elected leader, sending a full tablet report...
I20250905 07:49:53.958034 13837 heartbeater.cc:499] Master 127.8.229.126:33945 was elected leader, sending a full tablet report...
I20250905 07:49:54.001583 13911 heartbeater.cc:499] Master 127.8.229.126:33945 was elected leader, sending a full tablet report...
I20250905 07:49:54.027333 9109 test_util.cc:276] Using random seed: 358135614
I20250905 07:49:54.041494 13635 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41700:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:49:54.042115 13635 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:49:54.061300 13802 tablet_service.cc:1468] Processing CreateTablet for tablet c42c85dff4c54c7ebcec77d2c44c26ca (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION VALUES < 715827882
I20250905 07:49:54.061368 13801 tablet_service.cc:1468] Processing CreateTablet for tablet aaca5cf9fc86466d96318b9d94a788cb (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20250905 07:49:54.061672 13725 tablet_service.cc:1468] Processing CreateTablet for tablet 662c3c89e7824ee1a01ac9a7a7eb617f (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20250905 07:49:54.061806 13802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c42c85dff4c54c7ebcec77d2c44c26ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.061734 13726 tablet_service.cc:1468] Processing CreateTablet for tablet aaca5cf9fc86466d96318b9d94a788cb (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20250905 07:49:54.061859 13800 tablet_service.cc:1468] Processing CreateTablet for tablet 662c3c89e7824ee1a01ac9a7a7eb617f (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20250905 07:49:54.062063 13725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 662c3c89e7824ee1a01ac9a7a7eb617f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.061385 13727 tablet_service.cc:1468] Processing CreateTablet for tablet c42c85dff4c54c7ebcec77d2c44c26ca (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION VALUES < 715827882
I20250905 07:49:54.062254 13726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaca5cf9fc86466d96318b9d94a788cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.062433 13727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c42c85dff4c54c7ebcec77d2c44c26ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.064733 13800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 662c3c89e7824ee1a01ac9a7a7eb617f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.070016 13801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaca5cf9fc86466d96318b9d94a788cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.074443 13932 tablet_bootstrap.cc:492] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: Bootstrap starting.
I20250905 07:49:54.075502 13931 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469: Bootstrap starting.
I20250905 07:49:54.076332 13932 tablet_bootstrap.cc:654] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.077306 13931 tablet_bootstrap.cc:654] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.078383 13932 tablet_bootstrap.cc:492] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: No bootstrap required, opened a new log
I20250905 07:49:54.078523 13932 ts_tablet_manager.cc:1397] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20250905 07:49:54.079160 13931 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469: No bootstrap required, opened a new log
I20250905 07:49:54.079334 13931 ts_tablet_manager.cc:1397] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20250905 07:49:54.079241 13932 raft_consensus.cc:357] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.079306 13876 tablet_service.cc:1468] Processing CreateTablet for tablet c42c85dff4c54c7ebcec77d2c44c26ca (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION VALUES < 715827882
I20250905 07:49:54.079463 13875 tablet_service.cc:1468] Processing CreateTablet for tablet aaca5cf9fc86466d96318b9d94a788cb (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20250905 07:49:54.079896 13876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c42c85dff4c54c7ebcec77d2c44c26ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.080163 13875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaca5cf9fc86466d96318b9d94a788cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.080307 13931 raft_consensus.cc:357] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.080474 13931 raft_consensus.cc:383] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.080586 13931 raft_consensus.cc:738] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75f639992a44f4f8bfeb0a77ba4b469, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.080839 13931 consensus_queue.cc:260] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.081540 13931 ts_tablet_manager.cc:1428] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:54.081789 13931 tablet_bootstrap.cc:492] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469: Bootstrap starting.
I20250905 07:49:54.083324 13931 tablet_bootstrap.cc:654] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.079892 13874 tablet_service.cc:1468] Processing CreateTablet for tablet 662c3c89e7824ee1a01ac9a7a7eb617f (DEFAULT_TABLE table=test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20250905 07:49:54.084486 13874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 662c3c89e7824ee1a01ac9a7a7eb617f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.085198 13932 raft_consensus.cc:383] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.085283 13932 raft_consensus.cc:738] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca0a744b5f0c46758a3e58e324f37373, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.085762 13932 consensus_queue.cc:260] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.086687 13932 ts_tablet_manager.cc:1428] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20250905 07:49:54.087791 13932 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: Bootstrap starting.
I20250905 07:49:54.087867 13937 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: Bootstrap starting.
I20250905 07:49:54.089521 13932 tablet_bootstrap.cc:654] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.089522 13937 tablet_bootstrap.cc:654] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.094089 13931 tablet_bootstrap.cc:492] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469: No bootstrap required, opened a new log
I20250905 07:49:54.094213 13931 ts_tablet_manager.cc:1397] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20250905 07:49:54.095073 13931 raft_consensus.cc:357] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.095317 13931 raft_consensus.cc:383] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.095463 13931 raft_consensus.cc:738] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75f639992a44f4f8bfeb0a77ba4b469, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.095767 13931 consensus_queue.cc:260] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.096212 13931 ts_tablet_manager.cc:1428] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:49:54.096366 13932 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: No bootstrap required, opened a new log
I20250905 07:49:54.096490 13932 ts_tablet_manager.cc:1397] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20250905 07:49:54.096752 13937 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: No bootstrap required, opened a new log
I20250905 07:49:54.096863 13937 ts_tablet_manager.cc:1397] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20250905 07:49:54.097244 13932 raft_consensus.cc:357] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.097401 13932 raft_consensus.cc:383] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.097493 13932 raft_consensus.cc:738] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca0a744b5f0c46758a3e58e324f37373, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.097468 13937 raft_consensus.cc:357] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.097594 13937 raft_consensus.cc:383] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.097657 13937 raft_consensus.cc:738] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15a3c759c6e4b7d8aed7f7e7d6b220b, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.097730 13932 consensus_queue.cc:260] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.097810 13937 consensus_queue.cc:260] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.098170 13932 ts_tablet_manager.cc:1428] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:49:54.098161 13931 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469: Bootstrap starting.
I20250905 07:49:54.098490 13932 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: Bootstrap starting.
I20250905 07:49:54.100193 13932 tablet_bootstrap.cc:654] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.100515 13937 ts_tablet_manager.cc:1428] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250905 07:49:54.100770 13938 raft_consensus.cc:491] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:54.100891 13937 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: Bootstrap starting.
I20250905 07:49:54.100903 13938 raft_consensus.cc:513] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.101862 13938 leader_election.cc:290] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.103075 13937 tablet_bootstrap.cc:654] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.100188 13931 tablet_bootstrap.cc:654] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469: Neither blocks nor log segments found. Creating new log.
I20250905 07:49:54.106660 13937 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: No bootstrap required, opened a new log
I20250905 07:49:54.106891 13937 ts_tablet_manager.cc:1397] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20250905 07:49:54.107486 13931 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469: No bootstrap required, opened a new log
I20250905 07:49:54.107692 13931 ts_tablet_manager.cc:1397] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20250905 07:49:54.108011 13937 raft_consensus.cc:357] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.108325 13937 raft_consensus.cc:383] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.108458 13937 raft_consensus.cc:738] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15a3c759c6e4b7d8aed7f7e7d6b220b, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.108630 13931 raft_consensus.cc:357] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.108893 13931 raft_consensus.cc:383] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.109045 13931 raft_consensus.cc:738] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75f639992a44f4f8bfeb0a77ba4b469, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.108850 13937 consensus_queue.cc:260] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.109318 13931 consensus_queue.cc:260] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.110000 13931 ts_tablet_manager.cc:1428] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:49:54.110918 13935 raft_consensus.cc:491] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:54.111074 13935 raft_consensus.cc:513] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.113843 13935 leader_election.cc:290] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.113962 13935 raft_consensus.cc:491] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:54.114078 13935 raft_consensus.cc:513] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.114638 13935 leader_election.cc:290] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.114748 13935 raft_consensus.cc:491] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:54.114848 13935 raft_consensus.cc:513] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.115414 13935 leader_election.cc:290] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.118381 13937 ts_tablet_manager.cc:1428] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20250905 07:49:54.121248 13937 tablet_bootstrap.cc:492] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b: Bootstrap starting.
I20250905 07:49:54.123039 13941 raft_consensus.cc:491] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:54.123263 13941 raft_consensus.cc:513] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.130867 13938 raft_consensus.cc:491] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:54.131050 13938 raft_consensus.cc:513] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.131808 13938 leader_election.cc:290] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.139920 13941 leader_election.cc:290] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645), d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761)
I20250905 07:49:54.143380 13937 tablet_bootstrap.cc:654] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b: Neither blocks nor log segments found. Creating new log.
W20250905 07:49:54.150982 13763 tablet.cc:2378] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250905 07:49:54.161298 13937 tablet_bootstrap.cc:492] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b: No bootstrap required, opened a new log
I20250905 07:49:54.162658 13937 ts_tablet_manager.cc:1397] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.007s
I20250905 07:49:54.163992 13937 raft_consensus.cc:357] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.166154 13937 raft_consensus.cc:383] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.166536 13937 raft_consensus.cc:738] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15a3c759c6e4b7d8aed7f7e7d6b220b, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.166869 13937 consensus_queue.cc:260] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.170050 13937 ts_tablet_manager.cc:1428] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20250905 07:49:54.177764 13737 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca0a744b5f0c46758a3e58e324f37373" is_pre_election: true
I20250905 07:49:54.178164 13886 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" is_pre_election: true
I20250905 07:49:54.178335 13886 raft_consensus.cc:2466] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca0a744b5f0c46758a3e58e324f37373 in term 0.
I20250905 07:49:54.178355 13885 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" is_pre_election: true
I20250905 07:49:54.178544 13885 raft_consensus.cc:2466] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca0a744b5f0c46758a3e58e324f37373 in term 0.
W20250905 07:49:54.178980 13851 leader_election.cc:343] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:54.179308 13700 leader_election.cc:304] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, ca0a744b5f0c46758a3e58e324f37373; no voters:
I20250905 07:49:54.179673 13938 raft_consensus.cc:2802] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:54.179821 13938 raft_consensus.cc:491] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:54.179914 13938 raft_consensus.cc:3058] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:54.180856 13812 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" is_pre_election: true
I20250905 07:49:54.181016 13812 raft_consensus.cc:2466] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15a3c759c6e4b7d8aed7f7e7d6b220b in term 0.
I20250905 07:49:54.181392 13849 leader_election.cc:304] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, d75f639992a44f4f8bfeb0a77ba4b469; no voters: ca0a744b5f0c46758a3e58e324f37373
I20250905 07:49:54.181778 13941 raft_consensus.cc:2802] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:54.181856 13941 raft_consensus.cc:491] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:54.181926 13941 raft_consensus.cc:3058] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:54.179692 13700 leader_election.cc:304] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, ca0a744b5f0c46758a3e58e324f37373; no voters:
I20250905 07:49:54.182490 13737 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca0a744b5f0c46758a3e58e324f37373" is_pre_election: true
I20250905 07:49:54.183717 13938 raft_consensus.cc:513] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.184151 13737 raft_consensus.cc:2391] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d75f639992a44f4f8bfeb0a77ba4b469 in current term 1: Already voted for candidate ca0a744b5f0c46758a3e58e324f37373 in this term.
I20250905 07:49:54.184528 13941 raft_consensus.cc:513] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.185155 13941 leader_election.cc:290] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [CANDIDATE]: Term 1 election: Requested vote from peers ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645), d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761)
I20250905 07:49:54.185283 13938 leader_election.cc:290] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 election: Requested vote from peers d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.185387 13736 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca0a744b5f0c46758a3e58e324f37373" is_pre_election: true
I20250905 07:49:54.185882 13737 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca0a744b5f0c46758a3e58e324f37373"
I20250905 07:49:54.186020 13736 raft_consensus.cc:2466] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d75f639992a44f4f8bfeb0a77ba4b469 in term 0.
I20250905 07:49:54.185575 13735 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca0a744b5f0c46758a3e58e324f37373" is_pre_election: true
I20250905 07:49:54.186177 13812 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469"
I20250905 07:49:54.186300 13812 raft_consensus.cc:3058] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Advancing to term 1
W20250905 07:49:54.186352 13851 leader_election.cc:343] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:54.186542 13938 raft_consensus.cc:2802] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:54.186668 13938 raft_consensus.cc:491] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:54.186789 13938 raft_consensus.cc:3058] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:54.186806 13777 leader_election.cc:304] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca0a744b5f0c46758a3e58e324f37373, d75f639992a44f4f8bfeb0a77ba4b469; no voters:
I20250905 07:49:54.187093 13947 raft_consensus.cc:2802] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:49:54.187179 13947 raft_consensus.cc:491] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:49:54.187263 13947 raft_consensus.cc:3058] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Advancing to term 1
W20250905 07:49:54.187299 13777 leader_election.cc:343] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645): Illegal state: must be running to vote when last-logged opid is not known
I20250905 07:49:54.189049 13812 raft_consensus.cc:2466] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15a3c759c6e4b7d8aed7f7e7d6b220b in term 1.
I20250905 07:49:54.189157 13947 raft_consensus.cc:513] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.189432 13938 raft_consensus.cc:513] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.189497 13849 leader_election.cc:304] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, d75f639992a44f4f8bfeb0a77ba4b469; no voters: ca0a744b5f0c46758a3e58e324f37373
I20250905 07:49:54.189649 13947 leader_election.cc:290] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 election: Requested vote from peers ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.189925 13941 raft_consensus.cc:2802] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:54.190037 13938 leader_election.cc:290] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 election: Requested vote from peers d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.189985 13735 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca0a744b5f0c46758a3e58e324f37373"
I20250905 07:49:54.190241 13735 raft_consensus.cc:2391] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d75f639992a44f4f8bfeb0a77ba4b469 in current term 1: Already voted for candidate ca0a744b5f0c46758a3e58e324f37373 in this term.
I20250905 07:49:54.191075 13941 raft_consensus.cc:695] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 LEADER]: Becoming Leader. State: Replica: c15a3c759c6e4b7d8aed7f7e7d6b220b, State: Running, Role: LEADER
I20250905 07:49:54.191387 13941 consensus_queue.cc:237] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.193564 13885 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b"
I20250905 07:49:54.193724 13885 raft_consensus.cc:3058] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:54.193992 13811 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" is_pre_election: true
I20250905 07:49:54.194207 13811 raft_consensus.cc:2391] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca0a744b5f0c46758a3e58e324f37373 in current term 1: Already voted for candidate d75f639992a44f4f8bfeb0a77ba4b469 in this term.
I20250905 07:49:54.194226 13809 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469"
I20250905 07:49:54.194422 13809 raft_consensus.cc:2391] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ca0a744b5f0c46758a3e58e324f37373 in current term 1: Already voted for candidate d75f639992a44f4f8bfeb0a77ba4b469 in this term.
I20250905 07:49:54.195104 13810 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469"
I20250905 07:49:54.195314 13810 raft_consensus.cc:3058] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:54.193691 13886 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b"
I20250905 07:49:54.195546 13886 raft_consensus.cc:3058] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:54.195607 13932 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: No bootstrap required, opened a new log
I20250905 07:49:54.195736 13932 ts_tablet_manager.cc:1397] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: Time spent bootstrapping tablet: real 0.097s user 0.034s sys 0.051s
I20250905 07:49:54.196432 13885 raft_consensus.cc:2466] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca0a744b5f0c46758a3e58e324f37373 in term 1.
I20250905 07:49:54.196498 13932 raft_consensus.cc:357] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.196645 13932 raft_consensus.cc:383] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:49:54.196691 13932 raft_consensus.cc:738] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca0a744b5f0c46758a3e58e324f37373, State: Initialized, Role: FOLLOWER
I20250905 07:49:54.196830 13700 leader_election.cc:304] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, ca0a744b5f0c46758a3e58e324f37373; no voters:
I20250905 07:49:54.196887 13932 consensus_queue.cc:260] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.197057 13957 raft_consensus.cc:2802] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:54.197925 13810 raft_consensus.cc:2466] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca0a744b5f0c46758a3e58e324f37373 in term 1.
I20250905 07:49:54.198539 13886 raft_consensus.cc:2466] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca0a744b5f0c46758a3e58e324f37373 in term 1.
I20250905 07:49:54.198959 13700 leader_election.cc:304] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, ca0a744b5f0c46758a3e58e324f37373; no voters: d75f639992a44f4f8bfeb0a77ba4b469
I20250905 07:49:54.199066 13812 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" is_pre_election: true
I20250905 07:49:54.199126 13932 ts_tablet_manager.cc:1428] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:49:54.199231 13812 raft_consensus.cc:2374] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ca0a744b5f0c46758a3e58e324f37373 in term 1. Re-sending same reply.
I20250905 07:49:54.199316 13938 raft_consensus.cc:2802] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:49:54.199405 13938 raft_consensus.cc:695] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Becoming Leader. State: Replica: ca0a744b5f0c46758a3e58e324f37373, State: Running, Role: LEADER
I20250905 07:49:54.199777 13938 consensus_queue.cc:237] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.201084 13633 catalog_manager.cc:5582] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b reported cstate change: term changed from 0 to 1, leader changed from <none> to c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67). New cstate: current_term: 1 leader_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } health_report { overall_health: HEALTHY } } }
I20250905 07:49:54.201323 13957 raft_consensus.cc:695] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Becoming Leader. State: Replica: ca0a744b5f0c46758a3e58e324f37373, State: Running, Role: LEADER
I20250905 07:49:54.201577 13957 consensus_queue.cc:237] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.204265 13635 catalog_manager.cc:5582] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "ca0a744b5f0c46758a3e58e324f37373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:54.204468 13885 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" is_pre_election: true
I20250905 07:49:54.204655 13885 raft_consensus.cc:2391] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d75f639992a44f4f8bfeb0a77ba4b469 in current term 1: Already voted for candidate ca0a744b5f0c46758a3e58e324f37373 in this term.
I20250905 07:49:54.204668 13886 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" is_pre_election: true
I20250905 07:49:54.204872 13886 raft_consensus.cc:2391] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d75f639992a44f4f8bfeb0a77ba4b469 in current term 1: Already voted for candidate ca0a744b5f0c46758a3e58e324f37373 in this term.
I20250905 07:49:54.205273 13884 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" is_pre_election: true
I20250905 07:49:54.205281 13775 leader_election.cc:304] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d75f639992a44f4f8bfeb0a77ba4b469; no voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, ca0a744b5f0c46758a3e58e324f37373
I20250905 07:49:54.205659 13883 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aaca5cf9fc86466d96318b9d94a788cb" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b"
I20250905 07:49:54.205717 13775 leader_election.cc:304] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d75f639992a44f4f8bfeb0a77ba4b469; no voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, ca0a744b5f0c46758a3e58e324f37373
I20250905 07:49:54.205760 13947 raft_consensus.cc:2747] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:49:54.205873 13883 raft_consensus.cc:2391] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d75f639992a44f4f8bfeb0a77ba4b469 in current term 1: Already voted for candidate ca0a744b5f0c46758a3e58e324f37373 in this term.
I20250905 07:49:54.205927 13947 raft_consensus.cc:2747] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:49:54.206176 13775 leader_election.cc:304] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d75f639992a44f4f8bfeb0a77ba4b469; no voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, ca0a744b5f0c46758a3e58e324f37373
I20250905 07:49:54.206475 13935 raft_consensus.cc:2747] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250905 07:49:54.208940 13633 catalog_manager.cc:5582] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "ca0a744b5f0c46758a3e58e324f37373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:54.230296 13957 raft_consensus.cc:491] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:49:54.230515 13957 raft_consensus.cc:513] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.231088 13957 leader_election.cc:290] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66:40761), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.231600 13812 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" is_pre_election: true
I20250905 07:49:54.231696 13883 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c42c85dff4c54c7ebcec77d2c44c26ca" candidate_uuid: "ca0a744b5f0c46758a3e58e324f37373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" is_pre_election: true
I20250905 07:49:54.231797 13812 raft_consensus.cc:2391] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca0a744b5f0c46758a3e58e324f37373 in current term 1: Already voted for candidate c15a3c759c6e4b7d8aed7f7e7d6b220b in this term.
I20250905 07:49:54.232244 13700 leader_election.cc:304] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca0a744b5f0c46758a3e58e324f37373; no voters: c15a3c759c6e4b7d8aed7f7e7d6b220b, d75f639992a44f4f8bfeb0a77ba4b469
I20250905 07:49:54.232486 13957 raft_consensus.cc:3058] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:49:54.234820 13957 raft_consensus.cc:2747] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:49:54.246384 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:49:54.249037 13963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:54.249050 13964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:49:54.249681 13966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:49:54.250062 9109 server_base.cc:1047] running on GCE node
I20250905 07:49:54.251012 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:49:54.251084 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:49:54.251113 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058594251108 us; error 0 us; skew 500 ppm
I20250905 07:49:54.251256 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:49:54.252472 9109 webserver.cc:480] Webserver started at http://127.8.229.68:45595/ using document root <none> and password file <none>
I20250905 07:49:54.252705 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:49:54.252775 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:49:54.252916 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:49:54.253573 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "ecd14922096e43b48dc2d954ba7e477d"
format_stamp: "Formatted at 2025-09-05 07:49:54 on dist-test-slave-pgzt"
I20250905 07:49:54.256152 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:49:54.257755 13971 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:54.258122 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20250905 07:49:54.258206 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "ecd14922096e43b48dc2d954ba7e477d"
format_stamp: "Formatted at 2025-09-05 07:49:54 on dist-test-slave-pgzt"
I20250905 07:49:54.258296 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:49:54.265424 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:49:54.265892 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:49:54.266366 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:49:54.267117 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:49:54.267167 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:54.267211 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:49:54.267234 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:49:54.271075 13941 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250905 07:49:54.274168 13958 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20250905 07:49:54.274242 13958 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20250905 07:49:54.281675 13941 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:54.282016 13957 consensus_queue.cc:1035] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:54.284188 13957 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:54.289336 13957 consensus_queue.cc:1035] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:54.294426 13957 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:49:54.310421 14044 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:40721 every 8 connection(s)
I20250905 07:49:54.310418 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:40721
I20250905 07:49:54.318583 14045 heartbeater.cc:344] Connected to a master server at 127.8.229.126:33945
I20250905 07:49:54.318706 14045 heartbeater.cc:461] Registering TS with master...
I20250905 07:49:54.319006 14045 heartbeater.cc:507] Master 127.8.229.126:33945 requested a full tablet report, sending...
I20250905 07:49:54.319728 13633 ts_manager.cc:194] Registered new tserver with Master: ecd14922096e43b48dc2d954ba7e477d (127.8.229.68:40721)
I20250905 07:49:54.320422 13633 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41712
I20250905 07:49:54.796540 13688 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:54.809499 13735 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f"
dest_uuid: "ca0a744b5f0c46758a3e58e324f37373"
mode: GRACEFUL
new_leader_uuid: "d75f639992a44f4f8bfeb0a77ba4b469"
from {username='slave'} at 127.0.0.1:34618
I20250905 07:49:54.809720 13735 raft_consensus.cc:604] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Received request to transfer leadership to TS d75f639992a44f4f8bfeb0a77ba4b469
I20250905 07:49:54.810191 13688 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250905 07:49:54.810333 13688 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:54.810335 13883 consensus_queue.cc:237] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } }
I20250905 07:49:54.812529 13735 raft_consensus.cc:1273] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Refusing update from remote peer c15a3c759c6e4b7d8aed7f7e7d6b220b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:54.812660 13810 raft_consensus.cc:1273] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Refusing update from remote peer c15a3c759c6e4b7d8aed7f7e7d6b220b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:54.813125 14052 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:49:54.813473 13941 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:49:54.816387 13735 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
I20250905 07:49:54.816366 13810 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
W20250905 07:49:54.819703 13851 consensus_peers.cc:489] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b -> Peer ecd14922096e43b48dc2d954ba7e477d (127.8.229.68:40721): Couldn't send request to peer ecd14922096e43b48dc2d954ba7e477d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c42c85dff4c54c7ebcec77d2c44c26ca. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:54.821024 13633 catalog_manager.cc:5582] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 reported cstate change: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
I20250905 07:49:54.854107 13941 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
I20250905 07:49:54.863206 13735 consensus_queue.cc:237] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } }
I20250905 07:49:54.865020 13810 raft_consensus.cc:1273] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Refusing update from remote peer ca0a744b5f0c46758a3e58e324f37373: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:54.865159 13883 raft_consensus.cc:1273] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Refusing update from remote peer ca0a744b5f0c46758a3e58e324f37373: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:49:54.865720 13957 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:49:54.866035 13938 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:49:54.868685 13810 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
I20250905 07:49:54.868700 13883 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
W20250905 07:49:54.871919 13702 consensus_peers.cc:489] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 -> Peer ecd14922096e43b48dc2d954ba7e477d (127.8.229.68:40721): Couldn't send request to peer ecd14922096e43b48dc2d954ba7e477d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aaca5cf9fc86466d96318b9d94a788cb. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:49:54.887342 13634 catalog_manager.cc:5582] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 reported cstate change: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "ca0a744b5f0c46758a3e58e324f37373" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
I20250905 07:49:54.893662 14061 ts_tablet_manager.cc:927] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: Initiating tablet copy from peer c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.894225 14061 tablet_copy_client.cc:323] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:42045
I20250905 07:49:54.899526 13896 tablet_copy_service.cc:140] P c15a3c759c6e4b7d8aed7f7e7d6b220b: Received BeginTabletCopySession request for tablet c42c85dff4c54c7ebcec77d2c44c26ca from peer ecd14922096e43b48dc2d954ba7e477d ({username='slave'} at 127.0.0.1:49696)
I20250905 07:49:54.899670 13896 tablet_copy_service.cc:161] P c15a3c759c6e4b7d8aed7f7e7d6b220b: Beginning new tablet copy session on tablet c42c85dff4c54c7ebcec77d2c44c26ca from peer ecd14922096e43b48dc2d954ba7e477d at {username='slave'} at 127.0.0.1:49696: session id = ecd14922096e43b48dc2d954ba7e477d-c42c85dff4c54c7ebcec77d2c44c26ca
I20250905 07:49:54.901470 13896 tablet_copy_source_session.cc:215] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:54.902595 14061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c42c85dff4c54c7ebcec77d2c44c26ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.906813 14061 tablet_copy_client.cc:806] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:54.907025 14061 tablet_copy_client.cc:670] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:54.908375 14061 tablet_copy_client.cc:538] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:54.910696 14061 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: Bootstrap starting.
I20250905 07:49:54.915318 14061 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:54.915668 14061 tablet_bootstrap.cc:492] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: Bootstrap complete.
I20250905 07:49:54.915884 14061 ts_tablet_manager.cc:1397] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: Time spent bootstrapping tablet: real 0.005s user 0.007s sys 0.000s
I20250905 07:49:54.916460 14061 raft_consensus.cc:357] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } }
I20250905 07:49:54.916607 14061 raft_consensus.cc:738] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecd14922096e43b48dc2d954ba7e477d, State: Initialized, Role: LEARNER
I20250905 07:49:54.916776 14061 consensus_queue.cc:260] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } }
I20250905 07:49:54.917423 14061 ts_tablet_manager.cc:1428] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250905 07:49:54.917493 14045 heartbeater.cc:499] Master 127.8.229.126:33945 was elected leader, sending a full tablet report...
I20250905 07:49:54.918116 13896 tablet_copy_service.cc:342] P c15a3c759c6e4b7d8aed7f7e7d6b220b: Request end of tablet copy session ecd14922096e43b48dc2d954ba7e477d-c42c85dff4c54c7ebcec77d2c44c26ca received from {username='slave'} at 127.0.0.1:49696
I20250905 07:49:54.918205 13896 tablet_copy_service.cc:434] P c15a3c759c6e4b7d8aed7f7e7d6b220b: ending tablet copy session ecd14922096e43b48dc2d954ba7e477d-c42c85dff4c54c7ebcec77d2c44c26ca on tablet c42c85dff4c54c7ebcec77d2c44c26ca with peer ecd14922096e43b48dc2d954ba7e477d
I20250905 07:49:54.948211 14034 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } } }
I20250905 07:49:54.978801 14011 raft_consensus.cc:1215] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250905 07:49:54.981209 14061 ts_tablet_manager.cc:927] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: Initiating tablet copy from peer ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645)
I20250905 07:49:54.981920 14061 tablet_copy_client.cc:323] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.65:33645
I20250905 07:49:54.987465 13747 tablet_copy_service.cc:140] P ca0a744b5f0c46758a3e58e324f37373: Received BeginTabletCopySession request for tablet aaca5cf9fc86466d96318b9d94a788cb from peer ecd14922096e43b48dc2d954ba7e477d ({username='slave'} at 127.0.0.1:34638)
I20250905 07:49:54.987643 13747 tablet_copy_service.cc:161] P ca0a744b5f0c46758a3e58e324f37373: Beginning new tablet copy session on tablet aaca5cf9fc86466d96318b9d94a788cb from peer ecd14922096e43b48dc2d954ba7e477d at {username='slave'} at 127.0.0.1:34638: session id = ecd14922096e43b48dc2d954ba7e477d-aaca5cf9fc86466d96318b9d94a788cb
I20250905 07:49:54.990056 13747 tablet_copy_source_session.cc:215] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:49:54.991225 14061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaca5cf9fc86466d96318b9d94a788cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:49:54.994837 13957 raft_consensus.cc:991] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: : Instructing follower d75f639992a44f4f8bfeb0a77ba4b469 to start an election
I20250905 07:49:54.995018 13957 raft_consensus.cc:1079] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Signalling peer d75f639992a44f4f8bfeb0a77ba4b469 to start an election
I20250905 07:49:54.995692 13810 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f"
dest_uuid: "d75f639992a44f4f8bfeb0a77ba4b469"
from {username='slave'} at 127.0.0.1:57574
I20250905 07:49:54.995880 13810 raft_consensus.cc:491] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250905 07:49:54.995994 13810 raft_consensus.cc:3058] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:54.997421 14061 tablet_copy_client.cc:806] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Starting download of 0 data blocks...
I20250905 07:49:54.997645 14061 tablet_copy_client.cc:670] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Starting download of 1 WAL segments...
I20250905 07:49:54.998394 13810 raft_consensus.cc:513] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:54.999246 13735 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca0a744b5f0c46758a3e58e324f37373"
I20250905 07:49:54.999416 13735 raft_consensus.cc:3053] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Stepping down as leader of term 1
I20250905 07:49:54.999609 14061 tablet_copy_client.cc:538] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:49:54.999630 13735 raft_consensus.cc:738] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca0a744b5f0c46758a3e58e324f37373, State: Running, Role: LEADER
I20250905 07:49:54.999796 13810 leader_election.cc:290] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 2 election: Requested vote from peers ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645), c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045)
I20250905 07:49:54.999971 13735 consensus_queue.cc:260] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:55.000268 13735 raft_consensus.cc:3058] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:55.000442 13883 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "662c3c89e7824ee1a01ac9a7a7eb617f" candidate_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b"
I20250905 07:49:55.000648 13883 raft_consensus.cc:3058] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:49:55.002091 13735 raft_consensus.cc:2466] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d75f639992a44f4f8bfeb0a77ba4b469 in term 2.
I20250905 07:49:55.002501 13777 leader_election.cc:304] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca0a744b5f0c46758a3e58e324f37373, d75f639992a44f4f8bfeb0a77ba4b469; no voters:
I20250905 07:49:55.002781 14055 raft_consensus.cc:2802] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:49:55.003057 14061 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: Bootstrap starting.
I20250905 07:49:55.003778 14055 raft_consensus.cc:695] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 2 LEADER]: Becoming Leader. State: Replica: d75f639992a44f4f8bfeb0a77ba4b469, State: Running, Role: LEADER
I20250905 07:49:55.004011 13883 raft_consensus.cc:2466] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d75f639992a44f4f8bfeb0a77ba4b469 in term 2.
I20250905 07:49:55.004043 14055 consensus_queue.cc:237] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } }
I20250905 07:49:55.006011 13635 catalog_manager.cc:5582] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 reported cstate change: term changed from 1 to 2, leader changed from ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65) to d75f639992a44f4f8bfeb0a77ba4b469 (127.8.229.66). New cstate: current_term: 2 leader_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } health_report { overall_health: UNKNOWN } } }
I20250905 07:49:55.009389 14061 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:49:55.009713 14061 tablet_bootstrap.cc:492] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: Bootstrap complete.
I20250905 07:49:55.009938 14061 ts_tablet_manager.cc:1397] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20250905 07:49:55.010592 14061 raft_consensus.cc:357] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } }
I20250905 07:49:55.010803 14061 raft_consensus.cc:738] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecd14922096e43b48dc2d954ba7e477d, State: Initialized, Role: LEARNER
I20250905 07:49:55.011041 14061 consensus_queue.cc:260] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: true } }
I20250905 07:49:55.011590 14061 ts_tablet_manager.cc:1428] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20250905 07:49:55.012245 13747 tablet_copy_service.cc:342] P ca0a744b5f0c46758a3e58e324f37373: Request end of tablet copy session ecd14922096e43b48dc2d954ba7e477d-aaca5cf9fc86466d96318b9d94a788cb received from {username='slave'} at 127.0.0.1:34638
I20250905 07:49:55.012387 13747 tablet_copy_service.cc:434] P ca0a744b5f0c46758a3e58e324f37373: ending tablet copy session ecd14922096e43b48dc2d954ba7e477d-aaca5cf9fc86466d96318b9d94a788cb on tablet aaca5cf9fc86466d96318b9d94a788cb with peer ecd14922096e43b48dc2d954ba7e477d
I20250905 07:49:55.081700 13735 raft_consensus.cc:1273] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 2 FOLLOWER]: Refusing update from remote peer d75f639992a44f4f8bfeb0a77ba4b469: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:49:55.082264 14055 consensus_queue.cc:1035] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:49:55.088258 14011 raft_consensus.cc:1215] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250905 07:49:55.107116 13883 raft_consensus.cc:1273] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 2 FOLLOWER]: Refusing update from remote peer d75f639992a44f4f8bfeb0a77ba4b469: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:49:55.107643 14055 consensus_queue.cc:1035] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:49:55.110643 14058 raft_consensus.cc:1062] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: attempting to promote NON_VOTER ecd14922096e43b48dc2d954ba7e477d to VOTER
I20250905 07:49:55.111454 14058 consensus_queue.cc:237] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } }
I20250905 07:49:55.113401 14011 raft_consensus.cc:1273] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Refusing update from remote peer c15a3c759c6e4b7d8aed7f7e7d6b220b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:49:55.113464 13810 raft_consensus.cc:1273] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Refusing update from remote peer c15a3c759c6e4b7d8aed7f7e7d6b220b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:49:55.113422 13735 raft_consensus.cc:1273] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Refusing update from remote peer c15a3c759c6e4b7d8aed7f7e7d6b220b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:49:55.113921 13941 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:49:55.114266 13958 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:49:55.114539 14052 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:49:55.117853 13736 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.118757 14011 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.122646 13633 catalog_manager.cc:5582] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 reported cstate change: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.198343 13958 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.201745 13810 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } attrs { replace: true } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.222638 13884 consensus_queue.cc:237] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } }
I20250905 07:49:55.224344 14011 raft_consensus.cc:1273] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Refusing update from remote peer c15a3c759c6e4b7d8aed7f7e7d6b220b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:49:55.224452 13810 raft_consensus.cc:1273] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Refusing update from remote peer c15a3c759c6e4b7d8aed7f7e7d6b220b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:49:55.224877 14052 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:49:55.225199 14075 consensus_queue.cc:1035] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:49:55.227819 14011 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.228487 13810 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.233117 13634 catalog_manager.cc:5582] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 reported cstate change: config changed from index 3 to 4, VOTER ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65) evicted. New cstate: current_term: 1 leader_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.274340 14034 raft_consensus.cc:1062] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: attempting to promote NON_VOTER ecd14922096e43b48dc2d954ba7e477d to VOTER
I20250905 07:49:55.275053 14034 consensus_queue.cc:237] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } }
I20250905 07:49:55.276960 13810 raft_consensus.cc:1273] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Refusing update from remote peer ca0a744b5f0c46758a3e58e324f37373: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:49:55.277189 14011 raft_consensus.cc:1273] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 LEARNER]: Refusing update from remote peer ca0a744b5f0c46758a3e58e324f37373: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:49:55.276960 13883 raft_consensus.cc:1273] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Refusing update from remote peer ca0a744b5f0c46758a3e58e324f37373: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:49:55.277549 14066 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:49:55.277881 13960 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:49:55.278254 13957 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:49:55.281227 13884 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.281693 14011 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.305532 13725 tablet_service.cc:1515] Processing DeleteTablet for tablet c42c85dff4c54c7ebcec77d2c44c26ca with delete_type TABLET_DATA_TOMBSTONED (TS ca0a744b5f0c46758a3e58e324f37373 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34584
I20250905 07:49:55.307634 13632 catalog_manager.cc:5582] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b reported cstate change: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ca0a744b5f0c46758a3e58e324f37373" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.308241 14077 tablet_replica.cc:331] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: stopping tablet replica
I20250905 07:49:55.308768 14077 raft_consensus.cc:2241] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:55.309029 14077 raft_consensus.cc:2270] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:55.310102 14077 ts_tablet_manager.cc:1905] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:55.316002 14077 ts_tablet_manager.cc:1918] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250905 07:49:55.316161 14077 log.cc:1199] T c42c85dff4c54c7ebcec77d2c44c26ca P ca0a744b5f0c46758a3e58e324f37373: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/c42c85dff4c54c7ebcec77d2c44c26ca
I20250905 07:49:55.316861 13621 catalog_manager.cc:4928] TS ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65:33645): tablet c42c85dff4c54c7ebcec77d2c44c26ca (table test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]) successfully deleted
I20250905 07:49:55.324014 13960 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.327847 13810 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ecd14922096e43b48dc2d954ba7e477d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } attrs { replace: true } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.369822 13736 consensus_queue.cc:237] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } }
I20250905 07:49:55.371769 13810 raft_consensus.cc:1273] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Refusing update from remote peer ca0a744b5f0c46758a3e58e324f37373: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:49:55.371851 14011 raft_consensus.cc:1273] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Refusing update from remote peer ca0a744b5f0c46758a3e58e324f37373: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:49:55.372345 13938 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:49:55.372849 13938 consensus_queue.cc:1035] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:49:55.375475 13810 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.375538 14011 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.379163 13635 catalog_manager.cc:5582] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d reported cstate change: config changed from index 3 to 4, VOTER c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67) evicted. New cstate: current_term: 1 leader_uuid: "ca0a744b5f0c46758a3e58e324f37373" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.398456 14052 raft_consensus.cc:2953] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ca0a744b5f0c46758a3e58e324f37373 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "c15a3c759c6e4b7d8aed7f7e7d6b220b" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 42045 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.401746 13619 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet c42c85dff4c54c7ebcec77d2c44c26ca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:55.435693 13874 tablet_service.cc:1515] Processing DeleteTablet for tablet aaca5cf9fc86466d96318b9d94a788cb with delete_type TABLET_DATA_TOMBSTONED (TS c15a3c759c6e4b7d8aed7f7e7d6b220b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49662
I20250905 07:49:55.436635 14078 tablet_replica.cc:331] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: stopping tablet replica
I20250905 07:49:55.436851 14078 raft_consensus.cc:2241] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:49:55.437150 14078 raft_consensus.cc:2270] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:49:55.438019 14078 ts_tablet_manager.cc:1905] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:49:55.442524 14078 ts_tablet_manager.cc:1918] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250905 07:49:55.442647 14078 log.cc:1199] T aaca5cf9fc86466d96318b9d94a788cb P c15a3c759c6e4b7d8aed7f7e7d6b220b: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/aaca5cf9fc86466d96318b9d94a788cb
I20250905 07:49:55.443325 13619 catalog_manager.cc:4928] TS c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67:42045): tablet aaca5cf9fc86466d96318b9d94a788cb (table test-workload [id=69a7fcfcb9bb4b6c8213c89e44f2dca5]) successfully deleted
I20250905 07:49:55.490600 13938 raft_consensus.cc:2953] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c15a3c759c6e4b7d8aed7f7e7d6b220b (127.8.229.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ca0a744b5f0c46758a3e58e324f37373" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33645 } } peers { permanent_uuid: "d75f639992a44f4f8bfeb0a77ba4b469" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 40761 } } peers { permanent_uuid: "ecd14922096e43b48dc2d954ba7e477d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 40721 } attrs { promote: false } } }
I20250905 07:49:55.493506 13621 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet aaca5cf9fc86466d96318b9d94a788cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:49:56.810701 13688 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:56.811761 13688 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:56.811913 13688 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:49:58.812265 13688 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:49:58.813015 13688 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:49:58.813163 13688 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:50:00.813740 13688 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:00.815026 13688 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:00.815207 13688 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:50:02.815677 13688 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:02.816386 13688 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:02.816509 13688 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:50:03.366658 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:50:03.412878 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:03.413183 9109 tablet_replica.cc:331] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373: stopping tablet replica
I20250905 07:50:03.413365 9109 raft_consensus.cc:2241] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:03.413568 9109 raft_consensus.cc:2270] T 662c3c89e7824ee1a01ac9a7a7eb617f P ca0a744b5f0c46758a3e58e324f37373 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.414180 9109 tablet_replica.cc:331] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373: stopping tablet replica
I20250905 07:50:03.414319 9109 raft_consensus.cc:2241] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:03.414578 9109 raft_consensus.cc:2270] T aaca5cf9fc86466d96318b9d94a788cb P ca0a744b5f0c46758a3e58e324f37373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.428746 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:50:03.432091 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:50:03.441011 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:03.441254 9109 tablet_replica.cc:331] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469: stopping tablet replica
I20250905 07:50:03.441444 9109 raft_consensus.cc:2241] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:03.441679 9109 raft_consensus.cc:2270] T aaca5cf9fc86466d96318b9d94a788cb P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.442203 9109 tablet_replica.cc:331] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469: stopping tablet replica
I20250905 07:50:03.442303 9109 raft_consensus.cc:2241] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:03.442495 9109 raft_consensus.cc:2270] T c42c85dff4c54c7ebcec77d2c44c26ca P d75f639992a44f4f8bfeb0a77ba4b469 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.442868 9109 tablet_replica.cc:331] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469: stopping tablet replica
I20250905 07:50:03.442957 9109 raft_consensus.cc:2241] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:03.443181 9109 raft_consensus.cc:2270] T 662c3c89e7824ee1a01ac9a7a7eb617f P d75f639992a44f4f8bfeb0a77ba4b469 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.456799 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:50:03.459911 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:50:03.467911 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:03.468153 9109 tablet_replica.cc:331] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b: stopping tablet replica
I20250905 07:50:03.468333 9109 raft_consensus.cc:2241] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:03.468501 9109 raft_consensus.cc:2270] T 662c3c89e7824ee1a01ac9a7a7eb617f P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.469054 9109 tablet_replica.cc:331] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b: stopping tablet replica
I20250905 07:50:03.469188 9109 raft_consensus.cc:2241] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:03.469445 9109 raft_consensus.cc:2270] T c42c85dff4c54c7ebcec77d2c44c26ca P c15a3c759c6e4b7d8aed7f7e7d6b220b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.483242 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:50:03.486362 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:50:03.512569 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:03.512871 9109 tablet_replica.cc:331] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d: stopping tablet replica
I20250905 07:50:03.513059 9109 raft_consensus.cc:2241] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:03.513288 9109 raft_consensus.cc:2270] T aaca5cf9fc86466d96318b9d94a788cb P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.513859 9109 tablet_replica.cc:331] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d: stopping tablet replica
I20250905 07:50:03.513986 9109 raft_consensus.cc:2241] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:03.514182 9109 raft_consensus.cc:2270] T c42c85dff4c54c7ebcec77d2c44c26ca P ecd14922096e43b48dc2d954ba7e477d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.527642 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:50:03.530684 9109 master.cc:561] Master@127.8.229.126:33945 shutting down...
I20250905 07:50:03.537740 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:03.537962 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:03.538077 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 8f55ada7d81b46d497a0a9584354caa7: stopping tablet replica
I20250905 07:50:03.551133 9109 master.cc:583] Master@127.8.229.126:33945 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingUnstableCluster (10826 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor
I20250905 07:50:03.564038 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:45615
I20250905 07:50:03.564532 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:03.566614 14085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.566665 14084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.567291 14087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:03.567606 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:03.567792 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:03.567843 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:03.567880 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058603567877 us; error 0 us; skew 500 ppm
I20250905 07:50:03.568023 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:03.569069 9109 webserver.cc:480] Webserver started at http://127.8.229.126:33367/ using document root <none> and password file <none>
I20250905 07:50:03.569274 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:03.569330 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:03.569463 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:03.570068 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "621c9793d7194db89086ab3206719036"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.572461 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:50:03.573871 14092 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.574164 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:50:03.574254 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "621c9793d7194db89086ab3206719036"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.574368 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:03.585274 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:03.585772 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:03.597930 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:45615
I20250905 07:50:03.597947 14143 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:45615 every 8 connection(s)
I20250905 07:50:03.599236 14144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:03.603179 14144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036: Bootstrap starting.
I20250905 07:50:03.604502 14144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:03.606123 14144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036: No bootstrap required, opened a new log
I20250905 07:50:03.606840 14144 raft_consensus.cc:357] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621c9793d7194db89086ab3206719036" member_type: VOTER }
I20250905 07:50:03.607000 14144 raft_consensus.cc:383] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:03.607077 14144 raft_consensus.cc:738] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621c9793d7194db89086ab3206719036, State: Initialized, Role: FOLLOWER
I20250905 07:50:03.607280 14144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621c9793d7194db89086ab3206719036" member_type: VOTER }
I20250905 07:50:03.607491 14144 raft_consensus.cc:397] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:50:03.607564 14144 raft_consensus.cc:491] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:50:03.607641 14144 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:03.610080 14144 raft_consensus.cc:513] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621c9793d7194db89086ab3206719036" member_type: VOTER }
I20250905 07:50:03.610256 14144 leader_election.cc:304] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 621c9793d7194db89086ab3206719036; no voters:
I20250905 07:50:03.610611 14144 leader_election.cc:290] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:50:03.610728 14147 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:03.611196 14147 raft_consensus.cc:695] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 1 LEADER]: Becoming Leader. State: Replica: 621c9793d7194db89086ab3206719036, State: Running, Role: LEADER
I20250905 07:50:03.611397 14144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:50:03.611462 14147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621c9793d7194db89086ab3206719036" member_type: VOTER }
I20250905 07:50:03.612339 14148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "621c9793d7194db89086ab3206719036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621c9793d7194db89086ab3206719036" member_type: VOTER } }
I20250905 07:50:03.612370 14149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 621c9793d7194db89086ab3206719036. Latest consensus state: current_term: 1 leader_uuid: "621c9793d7194db89086ab3206719036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621c9793d7194db89086ab3206719036" member_type: VOTER } }
I20250905 07:50:03.612583 14149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:03.613077 14148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:03.616789 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:50:03.616765 14152 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:50:03.618060 14152 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:50:03.620656 14152 catalog_manager.cc:1349] Generated new cluster ID: 311ab6bf3e3a47c8aeddb7524d7de02d
I20250905 07:50:03.620718 14152 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:50:03.633877 14152 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:50:03.634675 14152 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:50:03.646591 14152 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036: Generated new TSK 0
I20250905 07:50:03.646836 14152 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:50:03.649327 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:03.651917 14165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.652184 14166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.652268 14168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:03.652684 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:03.652896 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:03.652941 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:03.652971 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058603652968 us; error 0 us; skew 500 ppm
I20250905 07:50:03.653084 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:03.654392 9109 webserver.cc:480] Webserver started at http://127.8.229.65:41735/ using document root <none> and password file <none>
I20250905 07:50:03.654575 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:03.654628 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:03.654724 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:03.655326 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "900b0ccf67734c56a0904cf11e8ce339"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.658015 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:03.659551 14173 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.659926 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:03.660012 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "900b0ccf67734c56a0904cf11e8ce339"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.660100 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:03.684906 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:03.685452 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:03.686034 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:03.686964 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:03.687021 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.687068 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:03.687098 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.701653 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:33685
I20250905 07:50:03.701684 14235 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:33685 every 8 connection(s)
I20250905 07:50:03.703341 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:03.706672 14240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.706888 14241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.708614 14243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:03.708856 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:03.709223 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:03.709293 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:03.709340 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058603709336 us; error 0 us; skew 500 ppm
I20250905 07:50:03.709489 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:03.710986 9109 webserver.cc:480] Webserver started at http://127.8.229.66:37553/ using document root <none> and password file <none>
I20250905 07:50:03.711218 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:03.711285 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:03.711395 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:03.711822 14236 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45615
I20250905 07:50:03.711958 14236 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:03.712186 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "6407241cebdd439cbd3f3b77ff3313ef"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.712371 14236 heartbeater.cc:507] Master 127.8.229.126:45615 requested a full tablet report, sending...
I20250905 07:50:03.713264 14109 ts_manager.cc:194] Registered new tserver with Master: 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685)
I20250905 07:50:03.714720 14109 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58628
I20250905 07:50:03.715214 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20250905 07:50:03.717028 14248 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.717399 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:03.717482 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "6407241cebdd439cbd3f3b77ff3313ef"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.717568 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:03.764884 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:03.765422 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:03.765940 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:03.766716 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:03.766767 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.766834 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:03.766875 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.779006 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:34769
I20250905 07:50:03.779034 14310 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:34769 every 8 connection(s)
I20250905 07:50:03.780383 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:03.782963 14314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.783656 14317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:03.783665 14315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:03.784922 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:03.785208 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:03.785270 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:03.785348 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058603785343 us; error 0 us; skew 500 ppm
I20250905 07:50:03.785498 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:03.786659 9109 webserver.cc:480] Webserver started at http://127.8.229.67:39463/ using document root <none> and password file <none>
I20250905 07:50:03.786862 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:03.786928 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:03.786988 14311 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45615
I20250905 07:50:03.787045 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:03.787106 14311 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:03.787449 14311 heartbeater.cc:507] Master 127.8.229.126:45615 requested a full tablet report, sending...
I20250905 07:50:03.787729 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "b8051c176cee4d8f81811a3a136eaedf"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.788216 14109 ts_manager.cc:194] Registered new tserver with Master: 6407241cebdd439cbd3f3b77ff3313ef (127.8.229.66:34769)
I20250905 07:50:03.788887 14109 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58632
I20250905 07:50:03.790139 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:03.791649 14322 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.792018 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:03.792121 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "b8051c176cee4d8f81811a3a136eaedf"
format_stamp: "Formatted at 2025-09-05 07:50:03 on dist-test-slave-pgzt"
I20250905 07:50:03.792215 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:03.810751 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:03.811333 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:03.811913 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:03.812806 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:03.812875 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.812944 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:03.812989 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:03.824226 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:38473
I20250905 07:50:03.824249 14384 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:38473 every 8 connection(s)
I20250905 07:50:03.830524 14385 heartbeater.cc:344] Connected to a master server at 127.8.229.126:45615
I20250905 07:50:03.830675 14385 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:03.830991 14385 heartbeater.cc:507] Master 127.8.229.126:45615 requested a full tablet report, sending...
I20250905 07:50:03.831772 14109 ts_manager.cc:194] Registered new tserver with Master: b8051c176cee4d8f81811a3a136eaedf (127.8.229.67:38473)
I20250905 07:50:03.831815 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006738243s
I20250905 07:50:03.832578 14109 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58644
I20250905 07:50:04.715962 14236 heartbeater.cc:499] Master 127.8.229.126:45615 was elected leader, sending a full tablet report...
I20250905 07:50:04.790014 14311 heartbeater.cc:499] Master 127.8.229.126:45615 was elected leader, sending a full tablet report...
I20250905 07:50:04.833738 14385 heartbeater.cc:499] Master 127.8.229.126:45615 was elected leader, sending a full tablet report...
I20250905 07:50:04.859206 9109 test_util.cc:276] Using random seed: 368967486
I20250905 07:50:04.870322 14109 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58652:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:50:04.871090 14109 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:50:04.912480 14201 tablet_service.cc:1468] Processing CreateTablet for tablet 92890fcfa1364f1f92d0df495be311c2 (DEFAULT_TABLE table=test-workload [id=5778c4b3fbc94d2fbf23c87dcef260ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:50:04.913046 14201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92890fcfa1364f1f92d0df495be311c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:04.913821 14350 tablet_service.cc:1468] Processing CreateTablet for tablet 92890fcfa1364f1f92d0df495be311c2 (DEFAULT_TABLE table=test-workload [id=5778c4b3fbc94d2fbf23c87dcef260ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:50:04.914189 14350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92890fcfa1364f1f92d0df495be311c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:04.914286 14276 tablet_service.cc:1468] Processing CreateTablet for tablet 92890fcfa1364f1f92d0df495be311c2 (DEFAULT_TABLE table=test-workload [id=5778c4b3fbc94d2fbf23c87dcef260ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20250905 07:50:04.914633 14276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92890fcfa1364f1f92d0df495be311c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:04.919378 14405 tablet_bootstrap.cc:492] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef: Bootstrap starting.
I20250905 07:50:04.919946 14406 tablet_bootstrap.cc:492] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: Bootstrap starting.
I20250905 07:50:04.921458 14405 tablet_bootstrap.cc:654] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:04.921682 14406 tablet_bootstrap.cc:654] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:04.923885 14405 tablet_bootstrap.cc:492] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef: No bootstrap required, opened a new log
I20250905 07:50:04.924117 14405 ts_tablet_manager.cc:1397] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20250905 07:50:04.925197 14405 raft_consensus.cc:357] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:04.925474 14405 raft_consensus.cc:383] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:04.925593 14405 raft_consensus.cc:738] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6407241cebdd439cbd3f3b77ff3313ef, State: Initialized, Role: FOLLOWER
I20250905 07:50:04.926149 14407 tablet_bootstrap.cc:492] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf: Bootstrap starting.
I20250905 07:50:04.925915 14405 consensus_queue.cc:260] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:04.927856 14405 ts_tablet_manager.cc:1428] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:50:04.928676 14407 tablet_bootstrap.cc:654] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:04.930567 14407 tablet_bootstrap.cc:492] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf: No bootstrap required, opened a new log
I20250905 07:50:04.930644 14406 tablet_bootstrap.cc:492] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: No bootstrap required, opened a new log
I20250905 07:50:04.930723 14407 ts_tablet_manager.cc:1397] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250905 07:50:04.930831 14406 ts_tablet_manager.cc:1397] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20250905 07:50:04.931594 14406 raft_consensus.cc:357] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:04.931802 14406 raft_consensus.cc:383] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:04.931917 14406 raft_consensus.cc:738] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900b0ccf67734c56a0904cf11e8ce339, State: Initialized, Role: FOLLOWER
I20250905 07:50:04.931656 14407 raft_consensus.cc:357] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:04.932193 14406 consensus_queue.cc:260] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:04.932348 14407 raft_consensus.cc:383] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:04.932459 14407 raft_consensus.cc:738] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8051c176cee4d8f81811a3a136eaedf, State: Initialized, Role: FOLLOWER
I20250905 07:50:04.932837 14407 consensus_queue.cc:260] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:04.932935 14406 ts_tablet_manager.cc:1428] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:04.933651 14407 ts_tablet_manager.cc:1428] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250905 07:50:05.007514 14412 raft_consensus.cc:491] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:05.007697 14412 raft_consensus.cc:513] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:05.008459 14412 leader_election.cc:290] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6407241cebdd439cbd3f3b77ff3313ef (127.8.229.66:34769), b8051c176cee4d8f81811a3a136eaedf (127.8.229.67:38473)
I20250905 07:50:05.013741 14360 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "92890fcfa1364f1f92d0df495be311c2" candidate_uuid: "900b0ccf67734c56a0904cf11e8ce339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8051c176cee4d8f81811a3a136eaedf" is_pre_election: true
I20250905 07:50:05.013788 14286 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "92890fcfa1364f1f92d0df495be311c2" candidate_uuid: "900b0ccf67734c56a0904cf11e8ce339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6407241cebdd439cbd3f3b77ff3313ef" is_pre_election: true
I20250905 07:50:05.013954 14360 raft_consensus.cc:2466] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 900b0ccf67734c56a0904cf11e8ce339 in term 0.
I20250905 07:50:05.013986 14286 raft_consensus.cc:2466] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 900b0ccf67734c56a0904cf11e8ce339 in term 0.
I20250905 07:50:05.014334 14177 leader_election.cc:304] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 900b0ccf67734c56a0904cf11e8ce339, b8051c176cee4d8f81811a3a136eaedf; no voters:
I20250905 07:50:05.014580 14412 raft_consensus.cc:2802] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:05.014660 14412 raft_consensus.cc:491] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:05.014748 14412 raft_consensus.cc:3058] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:05.016645 14412 raft_consensus.cc:513] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:05.017192 14412 leader_election.cc:290] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [CANDIDATE]: Term 1 election: Requested vote from peers 6407241cebdd439cbd3f3b77ff3313ef (127.8.229.66:34769), b8051c176cee4d8f81811a3a136eaedf (127.8.229.67:38473)
I20250905 07:50:05.017505 14286 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "92890fcfa1364f1f92d0df495be311c2" candidate_uuid: "900b0ccf67734c56a0904cf11e8ce339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6407241cebdd439cbd3f3b77ff3313ef"
I20250905 07:50:05.017552 14360 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "92890fcfa1364f1f92d0df495be311c2" candidate_uuid: "900b0ccf67734c56a0904cf11e8ce339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8051c176cee4d8f81811a3a136eaedf"
I20250905 07:50:05.017707 14360 raft_consensus.cc:3058] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:05.017709 14286 raft_consensus.cc:3058] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:05.020342 14360 raft_consensus.cc:2466] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 900b0ccf67734c56a0904cf11e8ce339 in term 1.
I20250905 07:50:05.020342 14286 raft_consensus.cc:2466] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 900b0ccf67734c56a0904cf11e8ce339 in term 1.
I20250905 07:50:05.020725 14177 leader_election.cc:304] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 900b0ccf67734c56a0904cf11e8ce339, b8051c176cee4d8f81811a3a136eaedf; no voters:
I20250905 07:50:05.020951 14412 raft_consensus.cc:2802] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:05.021327 14412 raft_consensus.cc:695] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 1 LEADER]: Becoming Leader. State: Replica: 900b0ccf67734c56a0904cf11e8ce339, State: Running, Role: LEADER
I20250905 07:50:05.021564 14412 consensus_queue.cc:237] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:05.023655 14109 catalog_manager.cc:5582] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 reported cstate change: term changed from 0 to 1, leader changed from <none> to 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "900b0ccf67734c56a0904cf11e8ce339" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:05.031823 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:50:05.056033 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:05.056490 9109 tablet_replica.cc:331] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: stopping tablet replica
I20250905 07:50:05.056798 9109 raft_consensus.cc:2241] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:05.057092 9109 pending_rounds.cc:70] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: Trying to abort 1 pending ops.
I20250905 07:50:05.057174 9109 pending_rounds.cc:77] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912046170218496 op_type: NO_OP noop_request { }
I20250905 07:50:05.057286 9109 raft_consensus.cc:2887] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:05.057376 9109 raft_consensus.cc:2270] T 92890fcfa1364f1f92d0df495be311c2 P 900b0ccf67734c56a0904cf11e8ce339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:05.071776 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:50:05.616662 14162 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:05.617017 14162 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:05.617123 14162 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:50:06.670603 14418 raft_consensus.cc:491] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:06.670794 14418 raft_consensus.cc:513] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:06.671602 14418 leader_election.cc:290] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685), b8051c176cee4d8f81811a3a136eaedf (127.8.229.67:38473)
W20250905 07:50:06.672554 14249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111) [suppressed 2 similar messages]
W20250905 07:50:06.673763 14249 leader_election.cc:336] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685): Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111)
I20250905 07:50:06.677446 14360 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "92890fcfa1364f1f92d0df495be311c2" candidate_uuid: "6407241cebdd439cbd3f3b77ff3313ef" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8051c176cee4d8f81811a3a136eaedf" is_pre_election: true
I20250905 07:50:06.677615 14360 raft_consensus.cc:2466] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6407241cebdd439cbd3f3b77ff3313ef in term 1.
I20250905 07:50:06.677938 14252 leader_election.cc:304] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6407241cebdd439cbd3f3b77ff3313ef, b8051c176cee4d8f81811a3a136eaedf; no voters: 900b0ccf67734c56a0904cf11e8ce339
I20250905 07:50:06.678171 14418 raft_consensus.cc:2802] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:06.678248 14418 raft_consensus.cc:491] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:06.678323 14418 raft_consensus.cc:3058] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:06.680092 14418 raft_consensus.cc:513] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:06.680591 14418 leader_election.cc:290] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [CANDIDATE]: Term 2 election: Requested vote from peers 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685), b8051c176cee4d8f81811a3a136eaedf (127.8.229.67:38473)
I20250905 07:50:06.681002 14360 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "92890fcfa1364f1f92d0df495be311c2" candidate_uuid: "6407241cebdd439cbd3f3b77ff3313ef" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8051c176cee4d8f81811a3a136eaedf"
I20250905 07:50:06.681164 14360 raft_consensus.cc:3058] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 1 FOLLOWER]: Advancing to term 2
W20250905 07:50:06.681671 14249 leader_election.cc:336] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685): Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111)
I20250905 07:50:06.682919 14360 raft_consensus.cc:2466] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6407241cebdd439cbd3f3b77ff3313ef in term 2.
I20250905 07:50:06.683230 14252 leader_election.cc:304] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6407241cebdd439cbd3f3b77ff3313ef, b8051c176cee4d8f81811a3a136eaedf; no voters: 900b0ccf67734c56a0904cf11e8ce339
I20250905 07:50:06.683466 14418 raft_consensus.cc:2802] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:06.683832 14418 raft_consensus.cc:695] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 2 LEADER]: Becoming Leader. State: Replica: 6407241cebdd439cbd3f3b77ff3313ef, State: Running, Role: LEADER
I20250905 07:50:06.684095 14418 consensus_queue.cc:237] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } }
I20250905 07:50:06.686478 14109 catalog_manager.cc:5582] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef reported cstate change: term changed from 1 to 2, leader changed from 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65) to 6407241cebdd439cbd3f3b77ff3313ef (127.8.229.66). New cstate: current_term: 2 leader_uuid: "6407241cebdd439cbd3f3b77ff3313ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6407241cebdd439cbd3f3b77ff3313ef" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 34769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "900b0ccf67734c56a0904cf11e8ce339" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 } health_report { overall_health: UNKNOWN } } }
W20250905 07:50:07.185904 14249 consensus_peers.cc:489] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef -> Peer 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685): Couldn't send request to peer 900b0ccf67734c56a0904cf11e8ce339. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:07.229507 14360 raft_consensus.cc:1273] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 2 FOLLOWER]: Refusing update from remote peer 6407241cebdd439cbd3f3b77ff3313ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250905 07:50:07.230106 14418 consensus_queue.cc:1035] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8051c176cee4d8f81811a3a136eaedf" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 38473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:07.617527 14162 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:07.618014 14162 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:07.618112 14162 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:50:09.618498 14162 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:09.619004 14162 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:09.619104 14162 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20250905 07:50:09.783174 14249 consensus_peers.cc:489] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef -> Peer 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685): Couldn't send request to peer 900b0ccf67734c56a0904cf11e8ce339. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250905 07:50:11.619519 14162 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:11.619935 14162 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:11.620016 14162 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20250905 07:50:12.264168 14249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20250905 07:50:12.265357 14249 consensus_peers.cc:489] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef -> Peer 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685): Couldn't send request to peer 900b0ccf67734c56a0904cf11e8ce339. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250905 07:50:13.620479 14162 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:13.620922 14162 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:13.621012 14162 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20250905 07:50:14.799418 14249 consensus_peers.cc:489] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef -> Peer 900b0ccf67734c56a0904cf11e8ce339 (127.8.229.65:33685): Couldn't send request to peer 900b0ccf67734c56a0904cf11e8ce339. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:33685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20250905 07:50:15.621423 14162 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250905 07:50:15.621873 14162 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250905 07:50:15.621970 14162 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20250905 07:50:16.139619 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:50:16.163403 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:16.163743 9109 tablet_replica.cc:331] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef: stopping tablet replica
I20250905 07:50:16.163934 9109 raft_consensus.cc:2241] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:16.164273 9109 raft_consensus.cc:2270] T 92890fcfa1364f1f92d0df495be311c2 P 6407241cebdd439cbd3f3b77ff3313ef [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:16.182382 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:50:16.185338 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:50:16.194698 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:16.194986 9109 tablet_replica.cc:331] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf: stopping tablet replica
I20250905 07:50:16.195209 9109 raft_consensus.cc:2241] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:16.195394 9109 raft_consensus.cc:2270] T 92890fcfa1364f1f92d0df495be311c2 P b8051c176cee4d8f81811a3a136eaedf [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:16.208957 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:50:16.211954 9109 master.cc:561] Master@127.8.229.126:45615 shutting down...
I20250905 07:50:16.220676 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:16.220928 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:16.221056 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 621c9793d7194db89086ab3206719036: stopping tablet replica
I20250905 07:50:16.234540 9109 master.cc:583] Master@127.8.229.126:45615 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor (12681 ms)
[ RUN ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering
I20250905 07:50:16.245246 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:39909
I20250905 07:50:16.245774 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:16.248119 14454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:16.248179 14455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:16.248847 9109 server_base.cc:1047] running on GCE node
W20250905 07:50:16.248970 14457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:16.249284 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:16.249368 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:16.249437 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058616249416 us; error 0 us; skew 500 ppm
I20250905 07:50:16.249599 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:16.250743 9109 webserver.cc:480] Webserver started at http://127.8.229.126:43447/ using document root <none> and password file <none>
I20250905 07:50:16.250967 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:16.251052 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:16.251190 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:16.251904 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "e87fdb8fadb54c939c0d40b76fd90411"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.254360 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:50:16.255892 14462 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.256271 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:50:16.256386 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "e87fdb8fadb54c939c0d40b76fd90411"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.256491 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:16.315567 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:16.316084 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:16.327641 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:39909
I20250905 07:50:16.327677 14513 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:39909 every 8 connection(s)
I20250905 07:50:16.328857 14514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:16.332793 14514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411: Bootstrap starting.
I20250905 07:50:16.334035 14514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:16.335624 14514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411: No bootstrap required, opened a new log
I20250905 07:50:16.336196 14514 raft_consensus.cc:357] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87fdb8fadb54c939c0d40b76fd90411" member_type: VOTER }
I20250905 07:50:16.336315 14514 raft_consensus.cc:383] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:16.336383 14514 raft_consensus.cc:738] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87fdb8fadb54c939c0d40b76fd90411, State: Initialized, Role: FOLLOWER
I20250905 07:50:16.336552 14514 consensus_queue.cc:260] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87fdb8fadb54c939c0d40b76fd90411" member_type: VOTER }
I20250905 07:50:16.336653 14514 raft_consensus.cc:397] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:50:16.336714 14514 raft_consensus.cc:491] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:50:16.336789 14514 raft_consensus.cc:3058] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:16.338436 14514 raft_consensus.cc:513] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87fdb8fadb54c939c0d40b76fd90411" member_type: VOTER }
I20250905 07:50:16.338598 14514 leader_election.cc:304] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e87fdb8fadb54c939c0d40b76fd90411; no voters:
I20250905 07:50:16.338946 14514 leader_election.cc:290] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:50:16.339044 14517 raft_consensus.cc:2802] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:16.339496 14517 raft_consensus.cc:695] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 1 LEADER]: Becoming Leader. State: Replica: e87fdb8fadb54c939c0d40b76fd90411, State: Running, Role: LEADER
I20250905 07:50:16.339704 14517 consensus_queue.cc:237] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87fdb8fadb54c939c0d40b76fd90411" member_type: VOTER }
I20250905 07:50:16.339735 14514 sys_catalog.cc:564] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:50:16.340431 14519 sys_catalog.cc:455] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e87fdb8fadb54c939c0d40b76fd90411. Latest consensus state: current_term: 1 leader_uuid: "e87fdb8fadb54c939c0d40b76fd90411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87fdb8fadb54c939c0d40b76fd90411" member_type: VOTER } }
I20250905 07:50:16.340651 14519 sys_catalog.cc:458] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:16.340899 14518 sys_catalog.cc:455] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e87fdb8fadb54c939c0d40b76fd90411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87fdb8fadb54c939c0d40b76fd90411" member_type: VOTER } }
I20250905 07:50:16.341099 14518 sys_catalog.cc:458] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:16.341841 14522 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:50:16.343087 14522 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:50:16.345253 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:50:16.345918 14522 catalog_manager.cc:1349] Generated new cluster ID: 178c1864549f40cd9f2a56e482a99fe8
I20250905 07:50:16.345993 14522 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:50:16.352653 14522 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:50:16.353410 14522 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:50:16.362567 14522 catalog_manager.cc:5955] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411: Generated new TSK 0
I20250905 07:50:16.362804 14522 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:50:16.378159 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:16.380824 14535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:16.381037 14536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:16.381830 14538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:16.382134 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:16.382359 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:16.382414 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:16.382458 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058616382453 us; error 0 us; skew 500 ppm
I20250905 07:50:16.382594 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:16.383688 9109 webserver.cc:480] Webserver started at http://127.8.229.65:45525/ using document root <none> and password file <none>
I20250905 07:50:16.383891 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:16.383960 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:16.384086 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:16.384720 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.387079 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:16.388680 14543 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.389001 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:16.389101 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.389231 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:16.398528 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:16.398995 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:16.399565 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:16.400377 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:16.400431 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.400499 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:16.400534 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.413179 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:38471
I20250905 07:50:16.413208 14605 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:38471 every 8 connection(s)
I20250905 07:50:16.414688 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:16.417373 14610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:16.418251 14611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:16.419391 14613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:16.420962 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:16.421347 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:16.421415 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:16.421458 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058616421453 us; error 0 us; skew 500 ppm
I20250905 07:50:16.421597 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:16.422755 9109 webserver.cc:480] Webserver started at http://127.8.229.66:37627/ using document root <none> and password file <none>
I20250905 07:50:16.422971 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:16.423040 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:16.423177 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:16.423810 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "875701b33779464db5850e62cadeed8d"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.426386 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:16.426913 14606 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39909
I20250905 07:50:16.427032 14606 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:16.427407 14606 heartbeater.cc:507] Master 127.8.229.126:39909 requested a full tablet report, sending...
I20250905 07:50:16.428323 14618 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.428375 14479 ts_manager.cc:194] Registered new tserver with Master: ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471)
I20250905 07:50:16.428649 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:16.428746 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "875701b33779464db5850e62cadeed8d"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.428845 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:16.429551 14479 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60138
I20250905 07:50:16.443111 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:16.443718 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:16.444232 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:16.444985 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:16.445042 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.445116 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:16.445161 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.457003 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:46237
I20250905 07:50:16.457032 14680 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:46237 every 8 connection(s)
I20250905 07:50:16.458459 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:16.460899 14684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:16.461246 14685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:16.462657 14687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:16.462967 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:16.463201 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:16.463255 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:16.463294 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058616463289 us; error 0 us; skew 500 ppm
I20250905 07:50:16.463476 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:16.464859 9109 webserver.cc:480] Webserver started at http://127.8.229.67:37997/ using document root <none> and password file <none>
I20250905 07:50:16.465075 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:16.465159 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:16.465281 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:16.465303 14681 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39909
I20250905 07:50:16.465415 14681 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:16.465735 14681 heartbeater.cc:507] Master 127.8.229.126:39909 requested a full tablet report, sending...
I20250905 07:50:16.465955 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "901c814643d34b0ea9c6d8de0398a537"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.466536 14479 ts_manager.cc:194] Registered new tserver with Master: 875701b33779464db5850e62cadeed8d (127.8.229.66:46237)
I20250905 07:50:16.467221 14479 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60150
I20250905 07:50:16.468588 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:16.469986 14692 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.470333 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:16.470417 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "901c814643d34b0ea9c6d8de0398a537"
format_stamp: "Formatted at 2025-09-05 07:50:16 on dist-test-slave-pgzt"
I20250905 07:50:16.470496 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:16.484263 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:16.484755 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:16.485220 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:16.486018 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:16.486071 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.486146 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:16.486191 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:16.498059 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:41661
I20250905 07:50:16.498083 14754 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:41661 every 8 connection(s)
I20250905 07:50:16.504278 14755 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39909
I20250905 07:50:16.504412 14755 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:16.504737 14755 heartbeater.cc:507] Master 127.8.229.126:39909 requested a full tablet report, sending...
I20250905 07:50:16.505528 14479 ts_manager.cc:194] Registered new tserver with Master: 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
I20250905 07:50:16.505642 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006756792s
I20250905 07:50:16.506295 14479 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60158
I20250905 07:50:17.430761 14606 heartbeater.cc:499] Master 127.8.229.126:39909 was elected leader, sending a full tablet report...
I20250905 07:50:17.468385 14681 heartbeater.cc:499] Master 127.8.229.126:39909 was elected leader, sending a full tablet report...
I20250905 07:50:17.507493 14755 heartbeater.cc:499] Master 127.8.229.126:39909 was elected leader, sending a full tablet report...
I20250905 07:50:17.533347 9109 test_util.cc:276] Using random seed: 381641627
I20250905 07:50:17.545848 14479 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60162:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:50:17.546530 14479 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:50:17.570621 14720 tablet_service.cc:1468] Processing CreateTablet for tablet b829203a2f7b48f2afa99fb7af68e4fb (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:17.571120 14720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b829203a2f7b48f2afa99fb7af68e4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.571274 14646 tablet_service.cc:1468] Processing CreateTablet for tablet b829203a2f7b48f2afa99fb7af68e4fb (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:17.571462 14717 tablet_service.cc:1468] Processing CreateTablet for tablet a370de2d89404c24877105026210ec42 (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:17.571736 14646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b829203a2f7b48f2afa99fb7af68e4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.571763 14644 tablet_service.cc:1468] Processing CreateTablet for tablet a4b6ffb18ea14de7b9d68fd05e40aab3 (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:17.571851 14717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a370de2d89404c24877105026210ec42. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.572120 14644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b6ffb18ea14de7b9d68fd05e40aab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.571130 14718 tablet_service.cc:1468] Processing CreateTablet for tablet a4b6ffb18ea14de7b9d68fd05e40aab3 (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:17.575906 14718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b6ffb18ea14de7b9d68fd05e40aab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.577842 14643 tablet_service.cc:1468] Processing CreateTablet for tablet a370de2d89404c24877105026210ec42 (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:17.578210 14643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a370de2d89404c24877105026210ec42. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.570706 14719 tablet_service.cc:1468] Processing CreateTablet for tablet 03838c6747a444f09c3738c5122f1dcf (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:17.579742 14719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03838c6747a444f09c3738c5122f1dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.579975 14569 tablet_service.cc:1468] Processing CreateTablet for tablet a4b6ffb18ea14de7b9d68fd05e40aab3 (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:17.580339 14569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b6ffb18ea14de7b9d68fd05e40aab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.579753 14568 tablet_service.cc:1468] Processing CreateTablet for tablet a370de2d89404c24877105026210ec42 (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:17.583842 14570 tablet_service.cc:1468] Processing CreateTablet for tablet 03838c6747a444f09c3738c5122f1dcf (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:17.584007 14568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a370de2d89404c24877105026210ec42. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.584204 14570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03838c6747a444f09c3738c5122f1dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.584895 14571 tablet_service.cc:1468] Processing CreateTablet for tablet b829203a2f7b48f2afa99fb7af68e4fb (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:17.585237 14571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b829203a2f7b48f2afa99fb7af68e4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.587620 14645 tablet_service.cc:1468] Processing CreateTablet for tablet 03838c6747a444f09c3738c5122f1dcf (DEFAULT_TABLE table=test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:17.588017 14645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03838c6747a444f09c3738c5122f1dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:17.603314 14776 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d: Bootstrap starting.
I20250905 07:50:17.606405 14776 tablet_bootstrap.cc:654] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.607759 14775 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: Bootstrap starting.
I20250905 07:50:17.607898 14777 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: Bootstrap starting.
I20250905 07:50:17.609239 14776 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d: No bootstrap required, opened a new log
I20250905 07:50:17.609364 14776 ts_tablet_manager.cc:1397] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250905 07:50:17.609544 14775 tablet_bootstrap.cc:654] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.609815 14777 tablet_bootstrap.cc:654] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.610261 14776 raft_consensus.cc:357] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.610562 14776 raft_consensus.cc:383] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.610647 14776 raft_consensus.cc:738] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875701b33779464db5850e62cadeed8d, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.610955 14776 consensus_queue.cc:260] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.611351 14775 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: No bootstrap required, opened a new log
I20250905 07:50:17.611555 14775 ts_tablet_manager.cc:1397] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:50:17.611774 14776 ts_tablet_manager.cc:1428] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:17.611872 14777 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: No bootstrap required, opened a new log
I20250905 07:50:17.612006 14777 ts_tablet_manager.cc:1397] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:50:17.612183 14776 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d: Bootstrap starting.
I20250905 07:50:17.612371 14775 raft_consensus.cc:357] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.612586 14775 raft_consensus.cc:383] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.612691 14775 raft_consensus.cc:738] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901c814643d34b0ea9c6d8de0398a537, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.612797 14777 raft_consensus.cc:357] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.613080 14777 raft_consensus.cc:383] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.612998 14775 consensus_queue.cc:260] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.613230 14777 raft_consensus.cc:738] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec88ae6b6e4f4b0da534f82bc15d27e7, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.613503 14777 consensus_queue.cc:260] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.614080 14776 tablet_bootstrap.cc:654] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.620596 14777 ts_tablet_manager.cc:1428] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent starting tablet: real 0.008s user 0.001s sys 0.006s
I20250905 07:50:17.621116 14777 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7: Bootstrap starting.
I20250905 07:50:17.622354 14776 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d: No bootstrap required, opened a new log
I20250905 07:50:17.622519 14776 ts_tablet_manager.cc:1397] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20250905 07:50:17.620373 14775 ts_tablet_manager.cc:1428] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: Time spent starting tablet: real 0.009s user 0.001s sys 0.006s
I20250905 07:50:17.622722 14777 tablet_bootstrap.cc:654] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.623414 14776 raft_consensus.cc:357] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.623740 14776 raft_consensus.cc:383] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.623893 14776 raft_consensus.cc:738] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875701b33779464db5850e62cadeed8d, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.624205 14776 consensus_queue.cc:260] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.624712 14776 ts_tablet_manager.cc:1428] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:50:17.624789 14775 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: Bootstrap starting.
I20250905 07:50:17.625356 14776 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: Bootstrap starting.
I20250905 07:50:17.626564 14775 tablet_bootstrap.cc:654] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.627651 14776 tablet_bootstrap.cc:654] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.631305 14777 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7: No bootstrap required, opened a new log
I20250905 07:50:17.631556 14777 ts_tablet_manager.cc:1397] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20250905 07:50:17.632627 14777 raft_consensus.cc:357] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.632926 14777 raft_consensus.cc:383] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.633066 14777 raft_consensus.cc:738] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec88ae6b6e4f4b0da534f82bc15d27e7, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.633471 14777 consensus_queue.cc:260] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.634428 14775 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: No bootstrap required, opened a new log
I20250905 07:50:17.634631 14775 ts_tablet_manager.cc:1397] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.005s
I20250905 07:50:17.634819 14777 ts_tablet_manager.cc:1428] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:50:17.635253 14777 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7: Bootstrap starting.
I20250905 07:50:17.635486 14775 raft_consensus.cc:357] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.635762 14775 raft_consensus.cc:383] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.635902 14775 raft_consensus.cc:738] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901c814643d34b0ea9c6d8de0398a537, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.636229 14775 consensus_queue.cc:260] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.636824 14775 ts_tablet_manager.cc:1428] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:50:17.637665 14777 tablet_bootstrap.cc:654] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.637861 14775 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: Bootstrap starting.
I20250905 07:50:17.639705 14775 tablet_bootstrap.cc:654] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.640215 14776 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: No bootstrap required, opened a new log
I20250905 07:50:17.640413 14776 ts_tablet_manager.cc:1397] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20250905 07:50:17.641393 14776 raft_consensus.cc:357] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.641706 14776 raft_consensus.cc:383] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.641861 14776 raft_consensus.cc:738] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875701b33779464db5850e62cadeed8d, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.642225 14776 consensus_queue.cc:260] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.643316 14777 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7: No bootstrap required, opened a new log
I20250905 07:50:17.643525 14777 ts_tablet_manager.cc:1397] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20250905 07:50:17.643628 14776 ts_tablet_manager.cc:1428] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:50:17.643942 14776 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d: Bootstrap starting.
I20250905 07:50:17.644316 14777 raft_consensus.cc:357] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.644544 14777 raft_consensus.cc:383] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.644685 14777 raft_consensus.cc:738] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec88ae6b6e4f4b0da534f82bc15d27e7, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.645020 14777 consensus_queue.cc:260] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.645774 14776 tablet_bootstrap.cc:654] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.646366 14777 ts_tablet_manager.cc:1428] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250905 07:50:17.646874 14777 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7: Bootstrap starting.
I20250905 07:50:17.648422 14775 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: No bootstrap required, opened a new log
I20250905 07:50:17.648576 14775 ts_tablet_manager.cc:1397] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20250905 07:50:17.648659 14777 tablet_bootstrap.cc:654] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.649211 14775 raft_consensus.cc:357] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.649435 14775 raft_consensus.cc:383] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.649533 14775 raft_consensus.cc:738] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901c814643d34b0ea9c6d8de0398a537, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.649861 14775 consensus_queue.cc:260] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.650391 14776 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d: No bootstrap required, opened a new log
I20250905 07:50:17.650512 14775 ts_tablet_manager.cc:1428] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:50:17.650571 14776 ts_tablet_manager.cc:1397] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250905 07:50:17.650606 14777 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7: No bootstrap required, opened a new log
I20250905 07:50:17.650831 14777 ts_tablet_manager.cc:1397] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20250905 07:50:17.650840 14775 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537: Bootstrap starting.
I20250905 07:50:17.651569 14776 raft_consensus.cc:357] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.651800 14776 raft_consensus.cc:383] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.651736 14777 raft_consensus.cc:357] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.651923 14776 raft_consensus.cc:738] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875701b33779464db5850e62cadeed8d, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.651998 14777 raft_consensus.cc:383] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.652132 14777 raft_consensus.cc:738] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec88ae6b6e4f4b0da534f82bc15d27e7, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.652191 14776 consensus_queue.cc:260] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.652370 14777 consensus_queue.cc:260] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.652875 14777 ts_tablet_manager.cc:1428] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:17.653491 14776 ts_tablet_manager.cc:1428] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20250905 07:50:17.656522 14775 tablet_bootstrap.cc:654] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:17.658084 14775 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537: No bootstrap required, opened a new log
I20250905 07:50:17.658236 14775 ts_tablet_manager.cc:1397] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20250905 07:50:17.658882 14775 raft_consensus.cc:357] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.659024 14775 raft_consensus.cc:383] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:17.659101 14775 raft_consensus.cc:738] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901c814643d34b0ea9c6d8de0398a537, State: Initialized, Role: FOLLOWER
I20250905 07:50:17.659291 14775 consensus_queue.cc:260] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.659745 14775 ts_tablet_manager.cc:1428] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20250905 07:50:17.705133 14782 raft_consensus.cc:491] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:17.705315 14782 raft_consensus.cc:513] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.706007 14782 leader_election.cc:290] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471), 875701b33779464db5850e62cadeed8d (127.8.229.66:46237)
I20250905 07:50:17.713408 14581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a370de2d89404c24877105026210ec42" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" is_pre_election: true
I20250905 07:50:17.713408 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a370de2d89404c24877105026210ec42" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d" is_pre_election: true
I20250905 07:50:17.713652 14581 raft_consensus.cc:2466] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 0.
I20250905 07:50:17.713696 14656 raft_consensus.cc:2466] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 0.
I20250905 07:50:17.714109 14695 leader_election.cc:304] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 901c814643d34b0ea9c6d8de0398a537, ec88ae6b6e4f4b0da534f82bc15d27e7; no voters:
I20250905 07:50:17.714403 14782 raft_consensus.cc:2802] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:17.714510 14782 raft_consensus.cc:491] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:17.714614 14782 raft_consensus.cc:3058] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.716542 14782 raft_consensus.cc:513] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.717118 14782 leader_election.cc:290] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 election: Requested vote from peers ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471), 875701b33779464db5850e62cadeed8d (127.8.229.66:46237)
I20250905 07:50:17.717553 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a370de2d89404c24877105026210ec42" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d"
I20250905 07:50:17.717527 14581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a370de2d89404c24877105026210ec42" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7"
I20250905 07:50:17.717738 14656 raft_consensus.cc:3058] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.717743 14581 raft_consensus.cc:3058] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.720479 14581 raft_consensus.cc:2466] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 1.
I20250905 07:50:17.720479 14656 raft_consensus.cc:2466] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 1.
I20250905 07:50:17.720888 14693 leader_election.cc:304] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 875701b33779464db5850e62cadeed8d, 901c814643d34b0ea9c6d8de0398a537; no voters:
I20250905 07:50:17.721180 14782 raft_consensus.cc:2802] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:17.721585 14782 raft_consensus.cc:695] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Becoming Leader. State: Replica: 901c814643d34b0ea9c6d8de0398a537, State: Running, Role: LEADER
I20250905 07:50:17.721839 14782 consensus_queue.cc:237] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.724153 14479 catalog_manager.cc:5582] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: term changed from 0 to 1, leader changed from <none> to 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:17.730510 14782 raft_consensus.cc:491] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:17.730686 14782 raft_consensus.cc:513] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.731339 14782 leader_election.cc:290] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 875701b33779464db5850e62cadeed8d (127.8.229.66:46237), ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471)
I20250905 07:50:17.731724 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b829203a2f7b48f2afa99fb7af68e4fb" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d" is_pre_election: true
I20250905 07:50:17.731722 14581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b829203a2f7b48f2afa99fb7af68e4fb" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" is_pre_election: true
I20250905 07:50:17.731928 14656 raft_consensus.cc:2466] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 0.
I20250905 07:50:17.731971 14581 raft_consensus.cc:2466] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 0.
I20250905 07:50:17.732288 14693 leader_election.cc:304] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 875701b33779464db5850e62cadeed8d, 901c814643d34b0ea9c6d8de0398a537; no voters:
I20250905 07:50:17.732532 14782 raft_consensus.cc:2802] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:17.732612 14782 raft_consensus.cc:491] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:17.732704 14782 raft_consensus.cc:3058] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.734648 14782 raft_consensus.cc:513] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.735183 14782 leader_election.cc:290] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 election: Requested vote from peers 875701b33779464db5850e62cadeed8d (127.8.229.66:46237), ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471)
I20250905 07:50:17.735563 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b829203a2f7b48f2afa99fb7af68e4fb" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d"
I20250905 07:50:17.735580 14581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "b829203a2f7b48f2afa99fb7af68e4fb" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7"
I20250905 07:50:17.735739 14581 raft_consensus.cc:3058] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.735745 14656 raft_consensus.cc:3058] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.737586 14656 raft_consensus.cc:2466] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 1.
I20250905 07:50:17.737591 14581 raft_consensus.cc:2466] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 1.
I20250905 07:50:17.737968 14695 leader_election.cc:304] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 875701b33779464db5850e62cadeed8d, 901c814643d34b0ea9c6d8de0398a537, ec88ae6b6e4f4b0da534f82bc15d27e7; no voters:
I20250905 07:50:17.738245 14782 raft_consensus.cc:2802] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:17.738365 14782 raft_consensus.cc:695] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Becoming Leader. State: Replica: 901c814643d34b0ea9c6d8de0398a537, State: Running, Role: LEADER
I20250905 07:50:17.738612 14782 consensus_queue.cc:237] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } }
I20250905 07:50:17.741035 14479 catalog_manager.cc:5582] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: term changed from 0 to 1, leader changed from <none> to 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:17.957091 14782 raft_consensus.cc:491] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:17.957302 14782 raft_consensus.cc:513] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.957989 14782 leader_election.cc:290] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471), 875701b33779464db5850e62cadeed8d (127.8.229.66:46237)
I20250905 07:50:17.958488 14581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a4b6ffb18ea14de7b9d68fd05e40aab3" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" is_pre_election: true
I20250905 07:50:17.958525 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a4b6ffb18ea14de7b9d68fd05e40aab3" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d" is_pre_election: true
I20250905 07:50:17.958698 14581 raft_consensus.cc:2466] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 0.
I20250905 07:50:17.958756 14656 raft_consensus.cc:2466] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 0.
I20250905 07:50:17.959110 14695 leader_election.cc:304] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 901c814643d34b0ea9c6d8de0398a537, ec88ae6b6e4f4b0da534f82bc15d27e7; no voters:
I20250905 07:50:17.959403 14782 raft_consensus.cc:2802] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:17.959517 14782 raft_consensus.cc:491] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:17.959623 14782 raft_consensus.cc:3058] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.961494 14782 raft_consensus.cc:513] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.962029 14782 leader_election.cc:290] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 election: Requested vote from peers ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471), 875701b33779464db5850e62cadeed8d (127.8.229.66:46237)
I20250905 07:50:17.962447 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a4b6ffb18ea14de7b9d68fd05e40aab3" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d"
I20250905 07:50:17.962414 14581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a4b6ffb18ea14de7b9d68fd05e40aab3" candidate_uuid: "901c814643d34b0ea9c6d8de0398a537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7"
I20250905 07:50:17.962625 14581 raft_consensus.cc:3058] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.962623 14656 raft_consensus.cc:3058] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:17.965337 14581 raft_consensus.cc:2466] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 1.
I20250905 07:50:17.965337 14656 raft_consensus.cc:2466] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901c814643d34b0ea9c6d8de0398a537 in term 1.
I20250905 07:50:17.965739 14695 leader_election.cc:304] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 901c814643d34b0ea9c6d8de0398a537, ec88ae6b6e4f4b0da534f82bc15d27e7; no voters:
I20250905 07:50:17.965998 14782 raft_consensus.cc:2802] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:17.966142 14782 raft_consensus.cc:695] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Becoming Leader. State: Replica: 901c814643d34b0ea9c6d8de0398a537, State: Running, Role: LEADER
I20250905 07:50:17.966408 14782 consensus_queue.cc:237] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:17.968869 14479 catalog_manager.cc:5582] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: term changed from 0 to 1, leader changed from <none> to 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:18.023211 14783 raft_consensus.cc:491] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:18.023480 14783 raft_consensus.cc:513] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:18.024369 14783 leader_election.cc:290] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 875701b33779464db5850e62cadeed8d (127.8.229.66:46237), 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
I20250905 07:50:18.032050 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "03838c6747a444f09c3738c5122f1dcf" candidate_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d" is_pre_election: true
I20250905 07:50:18.032066 14730 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "03838c6747a444f09c3738c5122f1dcf" candidate_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "901c814643d34b0ea9c6d8de0398a537" is_pre_election: true
I20250905 07:50:18.032303 14656 raft_consensus.cc:2466] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec88ae6b6e4f4b0da534f82bc15d27e7 in term 0.
I20250905 07:50:18.032336 14730 raft_consensus.cc:2466] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec88ae6b6e4f4b0da534f82bc15d27e7 in term 0.
I20250905 07:50:18.032675 14544 leader_election.cc:304] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 875701b33779464db5850e62cadeed8d, ec88ae6b6e4f4b0da534f82bc15d27e7; no voters:
I20250905 07:50:18.032938 14783 raft_consensus.cc:2802] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:18.033046 14783 raft_consensus.cc:491] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:18.033147 14783 raft_consensus.cc:3058] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:18.035208 14783 raft_consensus.cc:513] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:18.035861 14783 leader_election.cc:290] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [CANDIDATE]: Term 1 election: Requested vote from peers 875701b33779464db5850e62cadeed8d (127.8.229.66:46237), 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
I20250905 07:50:18.036237 14656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "03838c6747a444f09c3738c5122f1dcf" candidate_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875701b33779464db5850e62cadeed8d"
I20250905 07:50:18.036255 14730 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "03838c6747a444f09c3738c5122f1dcf" candidate_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "901c814643d34b0ea9c6d8de0398a537"
I20250905 07:50:18.036437 14656 raft_consensus.cc:3058] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:18.036444 14730 raft_consensus.cc:3058] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:18.038376 14656 raft_consensus.cc:2466] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec88ae6b6e4f4b0da534f82bc15d27e7 in term 1.
I20250905 07:50:18.038381 14730 raft_consensus.cc:2466] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec88ae6b6e4f4b0da534f82bc15d27e7 in term 1.
I20250905 07:50:18.038735 14547 leader_election.cc:304] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 901c814643d34b0ea9c6d8de0398a537, ec88ae6b6e4f4b0da534f82bc15d27e7; no voters:
I20250905 07:50:18.038986 14783 raft_consensus.cc:2802] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:18.039404 14783 raft_consensus.cc:695] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 LEADER]: Becoming Leader. State: Replica: ec88ae6b6e4f4b0da534f82bc15d27e7, State: Running, Role: LEADER
I20250905 07:50:18.039661 14783 consensus_queue.cc:237] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:18.042207 14479 catalog_manager.cc:5582] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:18.140902 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:50:18.148643 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:18.149034 9109 tablet_replica.cc:331] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7: stopping tablet replica
I20250905 07:50:18.149276 9109 raft_consensus.cc:2241] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:18.149477 9109 raft_consensus.cc:2270] T a4b6ffb18ea14de7b9d68fd05e40aab3 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:18.150168 9109 tablet_replica.cc:331] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7: stopping tablet replica
I20250905 07:50:18.150343 9109 raft_consensus.cc:2241] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:18.150497 9109 raft_consensus.cc:2270] T a370de2d89404c24877105026210ec42 P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:18.150929 9109 tablet_replica.cc:331] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: stopping tablet replica
I20250905 07:50:18.151103 9109 raft_consensus.cc:2241] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:18.151309 9109 pending_rounds.cc:70] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: Trying to abort 1 pending ops.
I20250905 07:50:18.151371 9109 pending_rounds.cc:77] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912099492687872 op_type: NO_OP noop_request { }
I20250905 07:50:18.151538 9109 raft_consensus.cc:2887] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:18.151648 9109 raft_consensus.cc:2270] T 03838c6747a444f09c3738c5122f1dcf P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:18.152218 9109 tablet_replica.cc:331] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7: stopping tablet replica
I20250905 07:50:18.152343 9109 raft_consensus.cc:2241] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:18.152490 9109 raft_consensus.cc:2270] T b829203a2f7b48f2afa99fb7af68e4fb P ec88ae6b6e4f4b0da534f82bc15d27e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:18.157295 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:50:18.192368 14782 consensus_queue.cc:1035] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250905 07:50:18.196367 14695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20250905 07:50:18.198449 14695 consensus_peers.cc:489] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:18.301378 14782 consensus_queue.cc:1035] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250905 07:50:18.306273 14695 consensus_peers.cc:489] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:18.513175 14806 consensus_queue.cc:1035] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250905 07:50:18.518049 14695 consensus_peers.cc:489] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:18.750641 14811 consensus_queue.cc:579] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Leader has been unable to successfully communicate with peer ec88ae6b6e4f4b0da534f82bc15d27e7 for more than 1 seconds (1.028s)
W20250905 07:50:18.753325 14477 catalog_manager.cc:5260] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a370de2d89404c24877105026210ec42 with cas_config_opid_index -1: no extra replica candidate found for tablet a370de2d89404c24877105026210ec42 (table test-workload [id=8e2f05f08b714f6a96c6efa44ea486a6]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20250905 07:50:19.093744 14811 consensus_queue.cc:579] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Leader has been unable to successfully communicate with peer ec88ae6b6e4f4b0da534f82bc15d27e7 for more than 1 seconds (1.127s)
I20250905 07:50:19.162178 14811 consensus_queue.cc:579] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Leader has been unable to successfully communicate with peer ec88ae6b6e4f4b0da534f82bc15d27e7 for more than 1 seconds (1.423s)
I20250905 07:50:19.174006 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:19.176446 14812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:19.176633 14813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:19.177788 14815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:19.177981 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:19.178212 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:19.178263 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:19.178299 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058619178296 us; error 0 us; skew 500 ppm
I20250905 07:50:19.178442 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:19.179757 9109 webserver.cc:480] Webserver started at http://127.8.229.68:39749/ using document root <none> and password file <none>
I20250905 07:50:19.179968 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:19.180028 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:19.180155 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:19.180763 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d"
format_stamp: "Formatted at 2025-09-05 07:50:19 on dist-test-slave-pgzt"
I20250905 07:50:19.183126 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20250905 07:50:19.184702 14820 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:19.185022 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:19.185132 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d"
format_stamp: "Formatted at 2025-09-05 07:50:19 on dist-test-slave-pgzt"
I20250905 07:50:19.185228 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:19.194447 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:19.194854 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:19.195334 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:19.196339 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:19.196403 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:19.196481 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:19.196528 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:19.208724 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:46267
I20250905 07:50:19.208750 14882 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:46267 every 8 connection(s)
I20250905 07:50:19.215364 14883 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39909
I20250905 07:50:19.215520 14883 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:19.215847 14883 heartbeater.cc:507] Master 127.8.229.126:39909 requested a full tablet report, sending...
I20250905 07:50:19.216591 14477 ts_manager.cc:194] Registered new tserver with Master: e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68:46267)
I20250905 07:50:19.217353 14477 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60178
I20250905 07:50:19.235004 14730 consensus_queue.cc:237] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:19.237114 14656 raft_consensus.cc:1273] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250905 07:50:19.237669 14811 consensus_queue.cc:1035] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250905 07:50:19.238282 14695 consensus_peers.cc:489] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:19.239660 14806 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
I20250905 07:50:19.240300 14656 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
W20250905 07:50:19.243474 14694 consensus_peers.cc:489] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 -> Peer e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68:46267): Couldn't send request to peer e93bd4bb55eb4f2cb60f1300cdf7881d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4b6ffb18ea14de7b9d68fd05e40aab3. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:19.243902 14466 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a4b6ffb18ea14de7b9d68fd05e40aab3 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 4)
I20250905 07:50:19.245280 14477 catalog_manager.cc:5582] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:19.277866 14730 consensus_queue.cc:237] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:19.279951 14656 raft_consensus.cc:1273] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250905 07:50:19.280124 14694 consensus_peers.cc:489] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 -> Peer e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68:46267): Couldn't send request to peer e93bd4bb55eb4f2cb60f1300cdf7881d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b829203a2f7b48f2afa99fb7af68e4fb. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:19.280464 14806 consensus_queue.cc:1035] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250905 07:50:19.280977 14695 consensus_peers.cc:489] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:19.282500 14782 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
I20250905 07:50:19.283051 14656 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
I20250905 07:50:19.285353 14466 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet b829203a2f7b48f2afa99fb7af68e4fb with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 3)
I20250905 07:50:19.286775 14479 catalog_manager.cc:5582] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250905 07:50:19.346128 14527 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver ec88ae6b6e4f4b0da534f82bc15d27e7 not available for placement
I20250905 07:50:19.404229 14730 consensus_queue.cc:237] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:19.406397 14656 raft_consensus.cc:1273] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250905 07:50:19.406808 14694 consensus_peers.cc:489] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 -> Peer e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68:46267): Couldn't send request to peer e93bd4bb55eb4f2cb60f1300cdf7881d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a370de2d89404c24877105026210ec42. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:19.406921 14782 consensus_queue.cc:1035] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250905 07:50:19.407272 14695 consensus_peers.cc:489] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:19.408998 14806 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
I20250905 07:50:19.409518 14656 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
I20250905 07:50:19.411756 14466 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a370de2d89404c24877105026210ec42 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 6)
I20250905 07:50:19.413038 14477 catalog_manager.cc:5582] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:19.597682 14889 raft_consensus.cc:491] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:19.597878 14889 raft_consensus.cc:513] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:19.598734 14889 leader_election.cc:290] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471), 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
W20250905 07:50:19.604941 14621 leader_election.cc:336] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
I20250905 07:50:19.608743 14730 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "03838c6747a444f09c3738c5122f1dcf" candidate_uuid: "875701b33779464db5850e62cadeed8d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "901c814643d34b0ea9c6d8de0398a537" is_pre_election: true
I20250905 07:50:19.608938 14730 raft_consensus.cc:2466] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 875701b33779464db5850e62cadeed8d in term 1.
I20250905 07:50:19.609294 14622 leader_election.cc:304] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 875701b33779464db5850e62cadeed8d, 901c814643d34b0ea9c6d8de0398a537; no voters: ec88ae6b6e4f4b0da534f82bc15d27e7
I20250905 07:50:19.609539 14889 raft_consensus.cc:2802] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:19.609612 14889 raft_consensus.cc:491] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:19.609685 14889 raft_consensus.cc:3058] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:19.611726 14889 raft_consensus.cc:513] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:19.612232 14889 leader_election.cc:290] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [CANDIDATE]: Term 2 election: Requested vote from peers ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471), 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
I20250905 07:50:19.612643 14730 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "03838c6747a444f09c3738c5122f1dcf" candidate_uuid: "875701b33779464db5850e62cadeed8d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "901c814643d34b0ea9c6d8de0398a537"
I20250905 07:50:19.612784 14730 raft_consensus.cc:3058] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Advancing to term 2
W20250905 07:50:19.613493 14621 leader_election.cc:336] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
I20250905 07:50:19.614699 14730 raft_consensus.cc:2466] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 875701b33779464db5850e62cadeed8d in term 2.
I20250905 07:50:19.615069 14622 leader_election.cc:304] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 875701b33779464db5850e62cadeed8d, 901c814643d34b0ea9c6d8de0398a537; no voters: ec88ae6b6e4f4b0da534f82bc15d27e7
I20250905 07:50:19.615286 14889 raft_consensus.cc:2802] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:19.615797 14889 raft_consensus.cc:695] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 LEADER]: Becoming Leader. State: Replica: 875701b33779464db5850e62cadeed8d, State: Running, Role: LEADER
I20250905 07:50:19.616046 14889 consensus_queue.cc:237] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } }
I20250905 07:50:19.618274 14479 catalog_manager.cc:5582] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d reported cstate change: term changed from 1 to 2, leader changed from ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) to 875701b33779464db5850e62cadeed8d (127.8.229.66). New cstate: current_term: 2 leader_uuid: "875701b33779464db5850e62cadeed8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:19.657842 14905 ts_tablet_manager.cc:927] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: Initiating tablet copy from peer 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
I20250905 07:50:19.658623 14905 tablet_copy_client.cc:323] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:41661
I20250905 07:50:19.664613 14740 tablet_copy_service.cc:140] P 901c814643d34b0ea9c6d8de0398a537: Received BeginTabletCopySession request for tablet b829203a2f7b48f2afa99fb7af68e4fb from peer e93bd4bb55eb4f2cb60f1300cdf7881d ({username='slave'} at 127.0.0.1:46320)
I20250905 07:50:19.664753 14740 tablet_copy_service.cc:161] P 901c814643d34b0ea9c6d8de0398a537: Beginning new tablet copy session on tablet b829203a2f7b48f2afa99fb7af68e4fb from peer e93bd4bb55eb4f2cb60f1300cdf7881d at {username='slave'} at 127.0.0.1:46320: session id = e93bd4bb55eb4f2cb60f1300cdf7881d-b829203a2f7b48f2afa99fb7af68e4fb
I20250905 07:50:19.666744 14740 tablet_copy_source_session.cc:215] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:19.667948 14905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b829203a2f7b48f2afa99fb7af68e4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:19.673766 14905 tablet_copy_client.cc:806] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:19.673980 14905 tablet_copy_client.cc:670] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:19.675768 14905 tablet_copy_client.cc:538] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:19.678197 14905 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap starting.
I20250905 07:50:19.682577 14905 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:19.682884 14905 tablet_bootstrap.cc:492] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap complete.
I20250905 07:50:19.683084 14905 ts_tablet_manager.cc:1397] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20250905 07:50:19.683758 14905 raft_consensus.cc:357] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:19.683908 14905 raft_consensus.cc:738] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e93bd4bb55eb4f2cb60f1300cdf7881d, State: Initialized, Role: LEARNER
I20250905 07:50:19.684087 14905 consensus_queue.cc:260] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:19.684854 14905 ts_tablet_manager.cc:1428] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:19.684902 14883 heartbeater.cc:499] Master 127.8.229.126:39909 was elected leader, sending a full tablet report...
I20250905 07:50:19.685601 14740 tablet_copy_service.cc:342] P 901c814643d34b0ea9c6d8de0398a537: Request end of tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-b829203a2f7b48f2afa99fb7af68e4fb received from {username='slave'} at 127.0.0.1:46320
I20250905 07:50:19.685707 14740 tablet_copy_service.cc:434] P 901c814643d34b0ea9c6d8de0398a537: ending tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-b829203a2f7b48f2afa99fb7af68e4fb on tablet b829203a2f7b48f2afa99fb7af68e4fb with peer e93bd4bb55eb4f2cb60f1300cdf7881d
I20250905 07:50:19.775861 14905 ts_tablet_manager.cc:927] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: Initiating tablet copy from peer 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
I20250905 07:50:19.776427 14905 tablet_copy_client.cc:323] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:41661
I20250905 07:50:19.777065 14740 tablet_copy_service.cc:140] P 901c814643d34b0ea9c6d8de0398a537: Received BeginTabletCopySession request for tablet a4b6ffb18ea14de7b9d68fd05e40aab3 from peer e93bd4bb55eb4f2cb60f1300cdf7881d ({username='slave'} at 127.0.0.1:46320)
I20250905 07:50:19.777194 14740 tablet_copy_service.cc:161] P 901c814643d34b0ea9c6d8de0398a537: Beginning new tablet copy session on tablet a4b6ffb18ea14de7b9d68fd05e40aab3 from peer e93bd4bb55eb4f2cb60f1300cdf7881d at {username='slave'} at 127.0.0.1:46320: session id = e93bd4bb55eb4f2cb60f1300cdf7881d-a4b6ffb18ea14de7b9d68fd05e40aab3
I20250905 07:50:19.779107 14740 tablet_copy_source_session.cc:215] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:19.780200 14905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b6ffb18ea14de7b9d68fd05e40aab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:19.784559 14905 tablet_copy_client.cc:806] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:19.784775 14905 tablet_copy_client.cc:670] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:19.786252 14905 tablet_copy_client.cc:538] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:19.788762 14905 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap starting.
I20250905 07:50:19.793215 14905 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:19.793642 14905 tablet_bootstrap.cc:492] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap complete.
I20250905 07:50:19.793861 14905 ts_tablet_manager.cc:1397] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250905 07:50:19.794498 14905 raft_consensus.cc:357] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:19.794647 14905 raft_consensus.cc:738] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e93bd4bb55eb4f2cb60f1300cdf7881d, State: Initialized, Role: LEARNER
I20250905 07:50:19.794809 14905 consensus_queue.cc:260] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:19.795248 14905 ts_tablet_manager.cc:1428] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:19.795989 14740 tablet_copy_service.cc:342] P 901c814643d34b0ea9c6d8de0398a537: Request end of tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-a4b6ffb18ea14de7b9d68fd05e40aab3 received from {username='slave'} at 127.0.0.1:46320
I20250905 07:50:19.796083 14740 tablet_copy_service.cc:434] P 901c814643d34b0ea9c6d8de0398a537: ending tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-a4b6ffb18ea14de7b9d68fd05e40aab3 on tablet a4b6ffb18ea14de7b9d68fd05e40aab3 with peer e93bd4bb55eb4f2cb60f1300cdf7881d
I20250905 07:50:20.006748 14905 ts_tablet_manager.cc:927] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: Initiating tablet copy from peer 901c814643d34b0ea9c6d8de0398a537 (127.8.229.67:41661)
I20250905 07:50:20.007499 14905 tablet_copy_client.cc:323] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:41661
I20250905 07:50:20.008198 14740 tablet_copy_service.cc:140] P 901c814643d34b0ea9c6d8de0398a537: Received BeginTabletCopySession request for tablet a370de2d89404c24877105026210ec42 from peer e93bd4bb55eb4f2cb60f1300cdf7881d ({username='slave'} at 127.0.0.1:46320)
I20250905 07:50:20.008389 14740 tablet_copy_service.cc:161] P 901c814643d34b0ea9c6d8de0398a537: Beginning new tablet copy session on tablet a370de2d89404c24877105026210ec42 from peer e93bd4bb55eb4f2cb60f1300cdf7881d at {username='slave'} at 127.0.0.1:46320: session id = e93bd4bb55eb4f2cb60f1300cdf7881d-a370de2d89404c24877105026210ec42
I20250905 07:50:20.012127 14740 tablet_copy_source_session.cc:215] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:20.013446 14905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a370de2d89404c24877105026210ec42. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:20.017927 14905 tablet_copy_client.cc:806] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:20.018126 14905 tablet_copy_client.cc:670] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:20.019544 14905 tablet_copy_client.cc:538] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:20.021914 14905 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap starting.
I20250905 07:50:20.026391 14905 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:20.026711 14905 tablet_bootstrap.cc:492] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap complete.
I20250905 07:50:20.026909 14905 ts_tablet_manager.cc:1397] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20250905 07:50:20.027606 14905 raft_consensus.cc:357] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:20.027755 14905 raft_consensus.cc:738] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e93bd4bb55eb4f2cb60f1300cdf7881d, State: Initialized, Role: LEARNER
I20250905 07:50:20.027915 14905 consensus_queue.cc:260] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:20.028354 14905 ts_tablet_manager.cc:1428] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250905 07:50:20.029091 14740 tablet_copy_service.cc:342] P 901c814643d34b0ea9c6d8de0398a537: Request end of tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-a370de2d89404c24877105026210ec42 received from {username='slave'} at 127.0.0.1:46320
I20250905 07:50:20.029199 14740 tablet_copy_service.cc:434] P 901c814643d34b0ea9c6d8de0398a537: ending tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-a370de2d89404c24877105026210ec42 on tablet a370de2d89404c24877105026210ec42 with peer e93bd4bb55eb4f2cb60f1300cdf7881d
I20250905 07:50:20.085633 14730 raft_consensus.cc:1273] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Refusing update from remote peer 875701b33779464db5850e62cadeed8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250905 07:50:20.086496 14889 consensus_queue.cc:1035] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250905 07:50:20.093483 14621 consensus_peers.cc:489] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:20.207782 14858 raft_consensus.cc:1215] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250905 07:50:20.220829 14858 raft_consensus.cc:1215] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250905 07:50:20.255663 14917 consensus_queue.cc:579] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Leader has been unable to successfully communicate with peer ec88ae6b6e4f4b0da534f82bc15d27e7 for more than 1 seconds (1.020s)
W20250905 07:50:20.346853 14527 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver ec88ae6b6e4f4b0da534f82bc15d27e7 not available for placement
I20250905 07:50:20.395618 14858 raft_consensus.cc:1215] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250905 07:50:20.594985 14917 raft_consensus.cc:1062] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: attempting to promote NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d to VOTER
I20250905 07:50:20.595741 14917 consensus_queue.cc:237] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:20.597651 14858 raft_consensus.cc:1273] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:50:20.598039 14656 raft_consensus.cc:1273] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:50:20.598188 14916 consensus_queue.cc:1035] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:20.598477 14919 consensus_queue.cc:1035] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250905 07:50:20.598922 14695 consensus_peers.cc:489] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:20.600666 14919 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.601248 14858 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.604223 14656 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.605187 14477 catalog_manager.cc:5582] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:20.610131 14730 consensus_queue.cc:237] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:20.611600 14656 raft_consensus.cc:1273] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:50:20.611598 14858 raft_consensus.cc:1273] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:50:20.612021 14896 consensus_queue.cc:1035] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:20.612192 14917 consensus_queue.cc:1035] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:20.614251 14916 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.614852 14656 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.615746 14858 raft_consensus.cc:2953] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.617404 14466 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet b829203a2f7b48f2afa99fb7af68e4fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:20.618503 14479 catalog_manager.cc:5582] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20250905 07:50:20.623860 14465 catalog_manager.cc:4671] TS ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b829203a2f7b48f2afa99fb7af68e4fb: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
I20250905 07:50:20.648015 14923 consensus_queue.cc:579] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Leader has been unable to successfully communicate with peer ec88ae6b6e4f4b0da534f82bc15d27e7 for more than 1 seconds (1.032s)
I20250905 07:50:20.652056 14656 consensus_queue.cc:237] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:20.653990 14730 raft_consensus.cc:1273] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Refusing update from remote peer 875701b33779464db5850e62cadeed8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:20.654707 14923 consensus_queue.cc:1035] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250905 07:50:20.654968 14621 consensus_peers.cc:489] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:20.656733 14929 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
I20250905 07:50:20.657384 14730 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } } }
W20250905 07:50:20.660679 14620 consensus_peers.cc:489] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d -> Peer e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68:46267): Couldn't send request to peer e93bd4bb55eb4f2cb60f1300cdf7881d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03838c6747a444f09c3738c5122f1dcf. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:20.661020 14463 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 03838c6747a444f09c3738c5122f1dcf with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20250905 07:50:20.662622 14477 catalog_manager.cc:5582] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d reported cstate change: config changed from index -1 to 2, NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "875701b33779464db5850e62cadeed8d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:20.730507 14896 raft_consensus.cc:1062] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: attempting to promote NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d to VOTER
I20250905 07:50:20.731189 14896 consensus_queue.cc:237] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:20.733218 14858 raft_consensus.cc:1273] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:50:20.733306 14656 raft_consensus.cc:1273] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:50:20.733736 14896 consensus_queue.cc:1035] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:20.733918 14919 consensus_queue.cc:1035] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250905 07:50:20.734151 14695 consensus_peers.cc:489] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:20.736469 14896 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.737044 14858 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.740824 14656 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.741667 14478 catalog_manager.cc:5582] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:20.746188 14730 consensus_queue.cc:237] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:20.747660 14858 raft_consensus.cc:1273] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:50:20.747660 14656 raft_consensus.cc:1273] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:50:20.748374 14917 consensus_queue.cc:1035] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:20.748556 14896 consensus_queue.cc:1035] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:20.750382 14917 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.750967 14656 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.751735 14858 raft_consensus.cc:2953] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.754264 14466 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet a4b6ffb18ea14de7b9d68fd05e40aab3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:20.755225 14479 catalog_manager.cc:5582] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d reported cstate change: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.929924 14896 consensus_queue.cc:579] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Leader has been unable to successfully communicate with peer ec88ae6b6e4f4b0da534f82bc15d27e7 for more than 1 seconds (1.525s)
I20250905 07:50:20.975335 14916 raft_consensus.cc:1062] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: attempting to promote NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d to VOTER
I20250905 07:50:20.976024 14916 consensus_queue.cc:237] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:20.977881 14858 raft_consensus.cc:1273] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 LEARNER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:50:20.978099 14656 raft_consensus.cc:1273] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250905 07:50:20.978374 14917 consensus_queue.cc:1035] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:20.978708 14919 consensus_queue.cc:1035] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250905 07:50:20.978868 14695 consensus_peers.cc:489] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:20.981086 14919 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.981670 14858 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.985659 14656 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.986305 14478 catalog_manager.cc:5582] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 reported cstate change: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:20.990514 14730 consensus_queue.cc:237] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:20.992012 14858 raft_consensus.cc:1273] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:50:20.992166 14656 raft_consensus.cc:1273] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Refusing update from remote peer 901c814643d34b0ea9c6d8de0398a537: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250905 07:50:20.992528 14919 consensus_queue.cc:1035] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:20.992729 14916 consensus_queue.cc:1035] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:20.994454 14917 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.995141 14858 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.995114 14656 raft_consensus.cc:2953] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:20.998220 14466 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet a370de2d89404c24877105026210ec42 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:20.999048 14479 catalog_manager.cc:5582] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d reported cstate change: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New cstate: current_term: 1 leader_uuid: "901c814643d34b0ea9c6d8de0398a537" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:21.167732 14941 ts_tablet_manager.cc:927] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: Initiating tablet copy from peer 875701b33779464db5850e62cadeed8d (127.8.229.66:46237)
I20250905 07:50:21.168648 14941 tablet_copy_client.cc:323] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:46237
I20250905 07:50:21.174289 14666 tablet_copy_service.cc:140] P 875701b33779464db5850e62cadeed8d: Received BeginTabletCopySession request for tablet 03838c6747a444f09c3738c5122f1dcf from peer e93bd4bb55eb4f2cb60f1300cdf7881d ({username='slave'} at 127.0.0.1:47616)
I20250905 07:50:21.174450 14666 tablet_copy_service.cc:161] P 875701b33779464db5850e62cadeed8d: Beginning new tablet copy session on tablet 03838c6747a444f09c3738c5122f1dcf from peer e93bd4bb55eb4f2cb60f1300cdf7881d at {username='slave'} at 127.0.0.1:47616: session id = e93bd4bb55eb4f2cb60f1300cdf7881d-03838c6747a444f09c3738c5122f1dcf
I20250905 07:50:21.176470 14666 tablet_copy_source_session.cc:215] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:21.177505 14941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03838c6747a444f09c3738c5122f1dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:21.181618 14941 tablet_copy_client.cc:806] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:21.181857 14941 tablet_copy_client.cc:670] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:21.183212 14941 tablet_copy_client.cc:538] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:21.185494 14941 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap starting.
I20250905 07:50:21.189827 14941 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:21.190162 14941 tablet_bootstrap.cc:492] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: Bootstrap complete.
I20250905 07:50:21.190371 14941 ts_tablet_manager.cc:1397] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20250905 07:50:21.190968 14941 raft_consensus.cc:357] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:21.191128 14941 raft_consensus.cc:738] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e93bd4bb55eb4f2cb60f1300cdf7881d, State: Initialized, Role: LEARNER
I20250905 07:50:21.191303 14941 consensus_queue.cc:260] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: true } }
I20250905 07:50:21.191768 14941 ts_tablet_manager.cc:1428] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20250905 07:50:21.192548 14666 tablet_copy_service.cc:342] P 875701b33779464db5850e62cadeed8d: Request end of tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-03838c6747a444f09c3738c5122f1dcf received from {username='slave'} at 127.0.0.1:47616
I20250905 07:50:21.192677 14666 tablet_copy_service.cc:434] P 875701b33779464db5850e62cadeed8d: ending tablet copy session e93bd4bb55eb4f2cb60f1300cdf7881d-03838c6747a444f09c3738c5122f1dcf on tablet 03838c6747a444f09c3738c5122f1dcf with peer e93bd4bb55eb4f2cb60f1300cdf7881d
W20250905 07:50:21.347890 14527 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver ec88ae6b6e4f4b0da534f82bc15d27e7 not available for placement
I20250905 07:50:21.586432 14858 raft_consensus.cc:1215] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
W20250905 07:50:21.654466 14465 catalog_manager.cc:4671] TS ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a370de2d89404c24877105026210ec42: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
I20250905 07:50:21.724248 14949 consensus_queue.cc:579] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Leader has been unable to successfully communicate with peer ec88ae6b6e4f4b0da534f82bc15d27e7 for more than 1 seconds (1.072s)
I20250905 07:50:21.979029 14949 raft_consensus.cc:1062] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: attempting to promote NON_VOTER e93bd4bb55eb4f2cb60f1300cdf7881d to VOTER
I20250905 07:50:21.979797 14949 consensus_queue.cc:237] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:21.981788 14730 raft_consensus.cc:1273] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Refusing update from remote peer 875701b33779464db5850e62cadeed8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:21.981823 14858 raft_consensus.cc:1273] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 LEARNER]: Refusing update from remote peer 875701b33779464db5850e62cadeed8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:21.982353 14951 consensus_queue.cc:1035] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:21.982616 14948 consensus_queue.cc:1035] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250905 07:50:21.982966 14621 consensus_peers.cc:489] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d -> Peer ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): Couldn't send request to peer ec88ae6b6e4f4b0da534f82bc15d27e7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:21.985483 14948 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:21.986186 14858 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:21.989184 14730 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:21.989869 14477 catalog_manager.cc:5582] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d reported cstate change: config changed from index 2 to 3, e93bd4bb55eb4f2cb60f1300cdf7881d (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "875701b33779464db5850e62cadeed8d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ec88ae6b6e4f4b0da534f82bc15d27e7" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:21.995306 14656 consensus_queue.cc:237] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } }
I20250905 07:50:21.997152 14858 raft_consensus.cc:1273] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 FOLLOWER]: Refusing update from remote peer 875701b33779464db5850e62cadeed8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:21.997285 14730 raft_consensus.cc:1273] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Refusing update from remote peer 875701b33779464db5850e62cadeed8d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:21.997650 14948 consensus_queue.cc:1035] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:21.997864 14951 consensus_queue.cc:1035] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:21.999838 14951 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:22.000564 14858 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:22.000689 14730 raft_consensus.cc:2953] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } } }
I20250905 07:50:22.003556 14463 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 03838c6747a444f09c3738c5122f1dcf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:22.005215 14478 catalog_manager.cc:5582] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d reported cstate change: config changed from index 3 to 4, VOTER ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65) evicted. New cstate: current_term: 2 leader_uuid: "875701b33779464db5850e62cadeed8d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "875701b33779464db5850e62cadeed8d" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "901c814643d34b0ea9c6d8de0398a537" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e93bd4bb55eb4f2cb60f1300cdf7881d" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 46267 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20250905 07:50:22.725033 14465 catalog_manager.cc:4671] TS ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 03838c6747a444f09c3738c5122f1dcf: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
W20250905 07:50:23.213892 14465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111) [suppressed 58 similar messages]
W20250905 07:50:24.347031 14465 catalog_manager.cc:4671] TS ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 03838c6747a444f09c3738c5122f1dcf: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
I20250905 07:50:26.252312 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:50:26.266455 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:26.266708 9109 tablet_replica.cc:331] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d: stopping tablet replica
I20250905 07:50:26.266889 9109 raft_consensus.cc:2241] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.267187 9109 raft_consensus.cc:2270] T b829203a2f7b48f2afa99fb7af68e4fb P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.267820 9109 tablet_replica.cc:331] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d: stopping tablet replica
I20250905 07:50:26.267982 9109 raft_consensus.cc:2241] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.268219 9109 raft_consensus.cc:2270] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.268651 9109 tablet_replica.cc:331] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d: stopping tablet replica
I20250905 07:50:26.268790 9109 raft_consensus.cc:2241] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:26.269059 9109 raft_consensus.cc:2270] T 03838c6747a444f09c3738c5122f1dcf P 875701b33779464db5850e62cadeed8d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.269462 9109 tablet_replica.cc:331] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d: stopping tablet replica
I20250905 07:50:26.269588 9109 raft_consensus.cc:2241] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.269786 9109 raft_consensus.cc:2270] T a370de2d89404c24877105026210ec42 P 875701b33779464db5850e62cadeed8d [term 1 FOLLOWER]: Raft consensus is shut down!
W20250905 07:50:26.286942 14693 consensus_peers.cc:489] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 -> Peer 875701b33779464db5850e62cadeed8d (127.8.229.66:46237): Couldn't send request to peer 875701b33779464db5850e62cadeed8d. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.66:46237: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:26.295991 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:50:26.299391 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:50:26.308933 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:26.309185 9109 tablet_replica.cc:331] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537: stopping tablet replica
I20250905 07:50:26.309392 9109 raft_consensus.cc:2241] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.309643 9109 raft_consensus.cc:2270] T 03838c6747a444f09c3738c5122f1dcf P 901c814643d34b0ea9c6d8de0398a537 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.310204 9109 tablet_replica.cc:331] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537: stopping tablet replica
I20250905 07:50:26.310331 9109 raft_consensus.cc:2241] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:26.310607 9109 raft_consensus.cc:2270] T a370de2d89404c24877105026210ec42 P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.311033 9109 tablet_replica.cc:331] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537: stopping tablet replica
I20250905 07:50:26.311169 9109 raft_consensus.cc:2241] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:26.311394 9109 raft_consensus.cc:2270] T b829203a2f7b48f2afa99fb7af68e4fb P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.311857 9109 tablet_replica.cc:331] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537: stopping tablet replica
I20250905 07:50:26.311968 9109 raft_consensus.cc:2241] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:26.312201 9109 raft_consensus.cc:2270] T a4b6ffb18ea14de7b9d68fd05e40aab3 P 901c814643d34b0ea9c6d8de0398a537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.327179 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:50:26.330775 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:50:26.340008 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:26.340229 9109 tablet_replica.cc:331] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d: stopping tablet replica
I20250905 07:50:26.340456 9109 raft_consensus.cc:2241] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.340655 9109 raft_consensus.cc:2270] T 03838c6747a444f09c3738c5122f1dcf P e93bd4bb55eb4f2cb60f1300cdf7881d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.341177 9109 tablet_replica.cc:331] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d: stopping tablet replica
I20250905 07:50:26.341305 9109 raft_consensus.cc:2241] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.341459 9109 raft_consensus.cc:2270] T a370de2d89404c24877105026210ec42 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.341833 9109 tablet_replica.cc:331] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d: stopping tablet replica
I20250905 07:50:26.341926 9109 raft_consensus.cc:2241] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.342073 9109 raft_consensus.cc:2270] T b829203a2f7b48f2afa99fb7af68e4fb P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.342412 9109 tablet_replica.cc:331] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d: stopping tablet replica
I20250905 07:50:26.342499 9109 raft_consensus.cc:2241] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:26.342634 9109 raft_consensus.cc:2270] T a4b6ffb18ea14de7b9d68fd05e40aab3 P e93bd4bb55eb4f2cb60f1300cdf7881d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:26.356550 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:50:26.360075 9109 master.cc:561] Master@127.8.229.126:39909 shutting down...
W20250905 07:50:26.444062 14465 catalog_manager.cc:4671] TS ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 03838c6747a444f09c3738c5122f1dcf: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
W20250905 07:50:29.031037 14465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20250905 07:50:29.032205 14465 catalog_manager.cc:4671] TS ec88ae6b6e4f4b0da534f82bc15d27e7 (127.8.229.65:38471): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b829203a2f7b48f2afa99fb7af68e4fb: Network error: Client connection negotiation failed: client connection to 127.8.229.65:38471: connect: Connection refused (error 111)
I20250905 07:50:29.959105 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:29.959424 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:29.959573 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P e87fdb8fadb54c939c0d40b76fd90411: stopping tablet replica
I20250905 07:50:29.975780 9109 master.cc:583] Master@127.8.229.126:39909 shutdown complete.
[ OK ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering (13756 ms)
[ RUN ] AutoRebalancerTest.TestHandlingFailedTservers
I20250905 07:50:30.001616 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:39689
I20250905 07:50:30.002137 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:30.004302 14968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:30.004429 14969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:30.005056 9109 server_base.cc:1047] running on GCE node
W20250905 07:50:30.005273 14971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:30.005584 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:30.005637 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:30.005668 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058630005665 us; error 0 us; skew 500 ppm
I20250905 07:50:30.005775 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:30.006829 9109 webserver.cc:480] Webserver started at http://127.8.229.126:37517/ using document root <none> and password file <none>
I20250905 07:50:30.007040 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:30.007102 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:30.007195 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:30.007812 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "4be40101869c4960b9809c639b9d0252"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.010174 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20250905 07:50:30.011607 14976 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.011926 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:30.012044 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "4be40101869c4960b9809c639b9d0252"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.012167 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:30.016974 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:30.017392 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:30.027276 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:39689
I20250905 07:50:30.027292 15027 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:39689 every 8 connection(s)
I20250905 07:50:30.028513 15028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:30.032460 15028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252: Bootstrap starting.
I20250905 07:50:30.033540 15028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:30.034931 15028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252: No bootstrap required, opened a new log
I20250905 07:50:30.035522 15028 raft_consensus.cc:357] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be40101869c4960b9809c639b9d0252" member_type: VOTER }
I20250905 07:50:30.035643 15028 raft_consensus.cc:383] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:30.035689 15028 raft_consensus.cc:738] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be40101869c4960b9809c639b9d0252, State: Initialized, Role: FOLLOWER
I20250905 07:50:30.035831 15028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be40101869c4960b9809c639b9d0252" member_type: VOTER }
I20250905 07:50:30.035917 15028 raft_consensus.cc:397] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:50:30.035955 15028 raft_consensus.cc:491] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:50:30.036003 15028 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:30.037719 15028 raft_consensus.cc:513] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be40101869c4960b9809c639b9d0252" member_type: VOTER }
I20250905 07:50:30.037868 15028 leader_election.cc:304] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4be40101869c4960b9809c639b9d0252; no voters:
I20250905 07:50:30.038178 15028 leader_election.cc:290] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:50:30.038282 15031 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:30.038736 15031 raft_consensus.cc:695] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 1 LEADER]: Becoming Leader. State: Replica: 4be40101869c4960b9809c639b9d0252, State: Running, Role: LEADER
I20250905 07:50:30.038940 15031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be40101869c4960b9809c639b9d0252" member_type: VOTER }
I20250905 07:50:30.039009 15028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:50:30.039702 15033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4be40101869c4960b9809c639b9d0252. Latest consensus state: current_term: 1 leader_uuid: "4be40101869c4960b9809c639b9d0252" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be40101869c4960b9809c639b9d0252" member_type: VOTER } }
I20250905 07:50:30.039834 15033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:30.040077 15032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4be40101869c4960b9809c639b9d0252" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be40101869c4960b9809c639b9d0252" member_type: VOTER } }
I20250905 07:50:30.040247 15032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:30.041496 15035 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:50:30.043466 15035 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:50:30.044171 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:50:30.046280 15035 catalog_manager.cc:1349] Generated new cluster ID: daaeec4102a44b55befbf417cc0ac8a7
I20250905 07:50:30.046350 15035 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:50:30.066625 15035 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:50:30.067415 15035 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:50:30.074877 15035 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252: Generated new TSK 0
I20250905 07:50:30.075116 15035 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:50:30.076643 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:30.079062 15049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:30.079291 15050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:30.079671 15052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:30.080224 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:30.080452 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:30.080507 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:30.080551 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058630080546 us; error 0 us; skew 500 ppm
I20250905 07:50:30.080684 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:30.081753 9109 webserver.cc:480] Webserver started at http://127.8.229.65:40519/ using document root <none> and password file <none>
I20250905 07:50:30.081959 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:30.082024 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:30.082162 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:30.082782 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.085278 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:50:30.086867 15057 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.087246 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:50:30.087337 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.087423 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:30.107678 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:30.108166 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:30.108688 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:30.109496 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:30.109547 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.109622 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:30.109659 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.122913 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:41883
I20250905 07:50:30.122974 15119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:41883 every 8 connection(s)
I20250905 07:50:30.124481 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:30.127385 15124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:30.127496 15125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:30.128232 9109 server_base.cc:1047] running on GCE node
W20250905 07:50:30.128587 15127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:30.128947 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:30.129002 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:30.129031 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058630129029 us; error 0 us; skew 500 ppm
I20250905 07:50:30.129189 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:30.130396 9109 webserver.cc:480] Webserver started at http://127.8.229.66:36569/ using document root <none> and password file <none>
I20250905 07:50:30.130627 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:30.130744 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:30.130878 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:30.131408 15120 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39689
I20250905 07:50:30.131551 15120 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:30.131621 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "96f1548c7bd14e87a78f184f668ce7f7"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.131882 15120 heartbeater.cc:507] Master 127.8.229.126:39689 requested a full tablet report, sending...
I20250905 07:50:30.132694 14993 ts_manager.cc:194] Registered new tserver with Master: 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883)
I20250905 07:50:30.133661 14993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60244
I20250905 07:50:30.134246 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:30.135901 15132 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.136217 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:30.136305 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "96f1548c7bd14e87a78f184f668ce7f7"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.136404 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:30.151310 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:30.151786 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:30.152292 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:30.153101 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:30.153165 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.153249 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:30.153294 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.164685 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:36935
I20250905 07:50:30.164716 15194 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:36935 every 8 connection(s)
I20250905 07:50:30.166201 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:30.168845 15198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:30.169054 15199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:30.169988 15201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:30.170501 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:30.170783 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:30.170897 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:30.170949 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058630170946 us; error 0 us; skew 500 ppm
I20250905 07:50:30.171082 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:30.172391 9109 webserver.cc:480] Webserver started at http://127.8.229.67:35667/ using document root <none> and password file <none>
I20250905 07:50:30.172626 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:30.172683 15195 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39689
I20250905 07:50:30.172694 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:30.172825 15195 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:30.172876 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:30.173132 15195 heartbeater.cc:507] Master 127.8.229.126:39689 requested a full tablet report, sending...
I20250905 07:50:30.173493 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "9e1f7067faba44858d1628228fd94c40"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.173959 14993 ts_manager.cc:194] Registered new tserver with Master: 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:30.174957 14993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60260
I20250905 07:50:30.176383 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20250905 07:50:30.177832 15206 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.178153 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:50:30.178232 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "9e1f7067faba44858d1628228fd94c40"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:30.178329 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:30.197604 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:30.198108 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:30.198624 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:30.199468 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:30.199537 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.199620 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:30.199661 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:30.210870 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:46381
I20250905 07:50:30.210897 15268 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:46381 every 8 connection(s)
I20250905 07:50:30.217445 15269 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39689
I20250905 07:50:30.217584 15269 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:30.217921 15269 heartbeater.cc:507] Master 127.8.229.126:39689 requested a full tablet report, sending...
I20250905 07:50:30.218664 14993 ts_manager.cc:194] Registered new tserver with Master: 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:30.219389 14993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60266
I20250905 07:50:30.219589 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00783976s
I20250905 07:50:31.135139 15120 heartbeater.cc:499] Master 127.8.229.126:39689 was elected leader, sending a full tablet report...
I20250905 07:50:31.176223 15195 heartbeater.cc:499] Master 127.8.229.126:39689 was elected leader, sending a full tablet report...
I20250905 07:50:31.220738 15269 heartbeater.cc:499] Master 127.8.229.126:39689 was elected leader, sending a full tablet report...
I20250905 07:50:31.247252 9109 test_util.cc:276] Using random seed: 395355534
I20250905 07:50:31.258559 14993 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60274:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:50:31.259261 14993 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:50:31.284652 15233 tablet_service.cc:1468] Processing CreateTablet for tablet 33e2555117254d15ba710c94aa648eec (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:31.285260 15233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e2555117254d15ba710c94aa648eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.285307 15085 tablet_service.cc:1468] Processing CreateTablet for tablet 46bbd12bea1345e888aae6cebac949b4 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:31.285665 15085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bbd12bea1345e888aae6cebac949b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.286325 15082 tablet_service.cc:1468] Processing CreateTablet for tablet 306950a7fe3940348981685598068091 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:31.286573 15159 tablet_service.cc:1468] Processing CreateTablet for tablet 33e2555117254d15ba710c94aa648eec (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:31.286765 15082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306950a7fe3940348981685598068091. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.286948 15159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e2555117254d15ba710c94aa648eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.287781 15083 tablet_service.cc:1468] Processing CreateTablet for tablet bebc2b83c54040d9a8dccb86347c42c8 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:31.288187 15083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bebc2b83c54040d9a8dccb86347c42c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.288951 15160 tablet_service.cc:1468] Processing CreateTablet for tablet 46bbd12bea1345e888aae6cebac949b4 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:31.289350 15160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bbd12bea1345e888aae6cebac949b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.292018 15158 tablet_service.cc:1468] Processing CreateTablet for tablet bebc2b83c54040d9a8dccb86347c42c8 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:31.292160 15157 tablet_service.cc:1468] Processing CreateTablet for tablet 306950a7fe3940348981685598068091 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:31.292485 15158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bebc2b83c54040d9a8dccb86347c42c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.292691 15157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306950a7fe3940348981685598068091. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.295064 15084 tablet_service.cc:1468] Processing CreateTablet for tablet 33e2555117254d15ba710c94aa648eec (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:31.295480 15084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e2555117254d15ba710c94aa648eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.284652 15234 tablet_service.cc:1468] Processing CreateTablet for tablet 46bbd12bea1345e888aae6cebac949b4 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:31.296180 15234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bbd12bea1345e888aae6cebac949b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.300393 15289 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:31.302423 15289 tablet_bootstrap.cc:654] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.307588 15232 tablet_service.cc:1468] Processing CreateTablet for tablet bebc2b83c54040d9a8dccb86347c42c8 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:31.308183 15232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bebc2b83c54040d9a8dccb86347c42c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.312197 15231 tablet_service.cc:1468] Processing CreateTablet for tablet 306950a7fe3940348981685598068091 (DEFAULT_TABLE table=test-workload [id=721383af9e624a91b8018ce5fe8646c8]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:31.312750 15231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306950a7fe3940348981685598068091. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:31.321870 15292 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:31.325254 15292 tablet_bootstrap.cc:654] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.333702 15292 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: No bootstrap required, opened a new log
I20250905 07:50:31.333987 15292 ts_tablet_manager.cc:1397] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20250905 07:50:31.335892 15292 raft_consensus.cc:357] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.336741 15290 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:31.336967 15292 raft_consensus.cc:383] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.337154 15292 raft_consensus.cc:738] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.337442 15292 consensus_queue.cc:260] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.337764 15289 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: No bootstrap required, opened a new log
I20250905 07:50:31.337906 15289 ts_tablet_manager.cc:1397] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.032s
I20250905 07:50:31.338312 15292 ts_tablet_manager.cc:1428] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20250905 07:50:31.338717 15289 raft_consensus.cc:357] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.338841 15290 tablet_bootstrap.cc:654] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.338925 15289 raft_consensus.cc:383] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.339028 15289 raft_consensus.cc:738] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.339041 15292 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:31.339320 15289 consensus_queue.cc:260] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.341044 15292 tablet_bootstrap.cc:654] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.346778 15289 ts_tablet_manager.cc:1428] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20250905 07:50:31.347064 15289 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:31.347090 15292 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: No bootstrap required, opened a new log
I20250905 07:50:31.347307 15292 ts_tablet_manager.cc:1397] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20250905 07:50:31.347952 15292 raft_consensus.cc:357] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.348136 15292 raft_consensus.cc:383] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.348218 15292 raft_consensus.cc:738] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.348433 15292 consensus_queue.cc:260] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.348780 15289 tablet_bootstrap.cc:654] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.348924 15292 ts_tablet_manager.cc:1428] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:31.349197 15292 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:31.351353 15292 tablet_bootstrap.cc:654] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.352602 15290 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: No bootstrap required, opened a new log
I20250905 07:50:31.352728 15290 ts_tablet_manager.cc:1397] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.005s
I20250905 07:50:31.353294 15290 raft_consensus.cc:357] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.353417 15290 raft_consensus.cc:383] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.353474 15290 raft_consensus.cc:738] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.353672 15290 consensus_queue.cc:260] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.354310 15290 ts_tablet_manager.cc:1428] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:50:31.354780 15290 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:31.355060 15292 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: No bootstrap required, opened a new log
I20250905 07:50:31.355194 15292 ts_tablet_manager.cc:1397] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20250905 07:50:31.355849 15292 raft_consensus.cc:357] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.355995 15292 raft_consensus.cc:383] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.356055 15292 raft_consensus.cc:738] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.356159 15290 tablet_bootstrap.cc:654] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.356254 15292 consensus_queue.cc:260] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.356703 15292 ts_tablet_manager.cc:1428] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250905 07:50:31.357017 15292 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:31.358678 15292 tablet_bootstrap.cc:654] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.360621 15292 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: No bootstrap required, opened a new log
I20250905 07:50:31.360774 15292 ts_tablet_manager.cc:1397] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20250905 07:50:31.361662 15292 raft_consensus.cc:357] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.361860 15292 raft_consensus.cc:383] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.361929 15292 raft_consensus.cc:738] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.362162 15292 consensus_queue.cc:260] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.362696 15292 ts_tablet_manager.cc:1428] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:50:31.363415 15289 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: No bootstrap required, opened a new log
I20250905 07:50:31.363574 15289 ts_tablet_manager.cc:1397] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20250905 07:50:31.364259 15289 raft_consensus.cc:357] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.364463 15289 raft_consensus.cc:383] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.364528 15290 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: No bootstrap required, opened a new log
I20250905 07:50:31.364559 15289 raft_consensus.cc:738] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.364656 15290 ts_tablet_manager.cc:1397] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20250905 07:50:31.364842 15289 consensus_queue.cc:260] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.365263 15289 ts_tablet_manager.cc:1428] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:31.365550 15289 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:31.365859 15290 raft_consensus.cc:357] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.366071 15290 raft_consensus.cc:383] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.366170 15290 raft_consensus.cc:738] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.366397 15290 consensus_queue.cc:260] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.366909 15290 ts_tablet_manager.cc:1428] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:50:31.367053 15289 tablet_bootstrap.cc:654] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.367215 15290 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:31.368747 15290 tablet_bootstrap.cc:654] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.370421 15290 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: No bootstrap required, opened a new log
I20250905 07:50:31.370515 15290 ts_tablet_manager.cc:1397] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250905 07:50:31.370991 15290 raft_consensus.cc:357] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.371129 15290 raft_consensus.cc:383] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.371192 15290 raft_consensus.cc:738] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.371377 15290 consensus_queue.cc:260] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.371814 15290 ts_tablet_manager.cc:1428] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250905 07:50:31.372052 15290 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:31.372193 15289 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: No bootstrap required, opened a new log
I20250905 07:50:31.372319 15289 ts_tablet_manager.cc:1397] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20250905 07:50:31.372871 15289 raft_consensus.cc:357] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.372985 15289 raft_consensus.cc:383] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.373019 15289 raft_consensus.cc:738] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.373293 15294 raft_consensus.cc:491] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:31.373492 15290 tablet_bootstrap.cc:654] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.373422 15294 raft_consensus.cc:513] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.373308 15289 consensus_queue.cc:260] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.373914 15289 ts_tablet_manager.cc:1428] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:31.374142 15289 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:31.374375 15294 leader_election.cc:290] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
W20250905 07:50:31.375250 15121 tablet.cc:2378] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250905 07:50:31.375730 15289 tablet_bootstrap.cc:654] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:31.381721 15294 raft_consensus.cc:491] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:31.381947 15294 raft_consensus.cc:513] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.382845 15294 leader_election.cc:290] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:31.396893 15290 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: No bootstrap required, opened a new log
I20250905 07:50:31.396917 15289 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: No bootstrap required, opened a new log
I20250905 07:50:31.397136 15290 ts_tablet_manager.cc:1397] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.014s
I20250905 07:50:31.397159 15289 ts_tablet_manager.cc:1397] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.007s
I20250905 07:50:31.397866 15290 raft_consensus.cc:357] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.398006 15289 raft_consensus.cc:357] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.398221 15289 raft_consensus.cc:383] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.398332 15289 raft_consensus.cc:738] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.398583 15289 consensus_queue.cc:260] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.398026 15290 raft_consensus.cc:383] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:31.399060 15290 raft_consensus.cc:738] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:31.399211 15289 ts_tablet_manager.cc:1428] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:50:31.399483 15290 consensus_queue.cc:260] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.399717 15300 raft_consensus.cc:491] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:31.399853 15300 raft_consensus.cc:513] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.399976 15169 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7" is_pre_election: true
I20250905 07:50:31.400149 15169 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 0.
I20250905 07:50:31.400188 15095 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" is_pre_election: true
I20250905 07:50:31.399909 15170 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7" is_pre_election: true
I20250905 07:50:31.400421 15095 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 0.
I20250905 07:50:31.400467 15170 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 0.
I20250905 07:50:31.400852 15300 leader_election.cc:290] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:31.400800 15094 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" is_pre_election: true
I20250905 07:50:31.401296 15094 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 0.
I20250905 07:50:31.400894 15209 leader_election.cc:304] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:31.401669 15294 raft_consensus.cc:2802] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:31.401811 15294 raft_consensus.cc:491] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:31.401919 15294 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.403029 15290 ts_tablet_manager.cc:1428] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20250905 07:50:31.403093 15296 raft_consensus.cc:491] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:31.403240 15296 raft_consensus.cc:513] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.404055 15296 leader_election.cc:290] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935), 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:31.405311 15208 leader_election.cc:304] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96f1548c7bd14e87a78f184f668ce7f7, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:31.405419 15294 raft_consensus.cc:513] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.406145 15294 leader_election.cc:290] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 election: Requested vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:31.408474 15094 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
I20250905 07:50:31.408646 15094 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.408190 15312 raft_consensus.cc:2802] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:31.409186 15312 raft_consensus.cc:491] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:31.409340 15312 raft_consensus.cc:3058] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.410547 15170 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7"
I20250905 07:50:31.410746 15170 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.411861 15094 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 1.
I20250905 07:50:31.412132 15312 raft_consensus.cc:513] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.412374 15209 leader_election.cc:304] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:31.412652 15294 raft_consensus.cc:2802] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:31.412823 15312 leader_election.cc:290] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 election: Requested vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:31.413872 15094 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
I20250905 07:50:31.414194 15170 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 1.
I20250905 07:50:31.414414 15094 raft_consensus.cc:3058] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.414983 15170 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7"
I20250905 07:50:31.415221 15170 raft_consensus.cc:3058] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.418632 15094 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 1.
I20250905 07:50:31.418927 15170 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 1.
I20250905 07:50:31.419916 15209 leader_election.cc:304] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:31.420254 15170 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7" is_pre_election: true
I20250905 07:50:31.420493 15170 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 0.
I20250905 07:50:31.420702 15294 raft_consensus.cc:695] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 LEADER]: Becoming Leader. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Running, Role: LEADER
I20250905 07:50:31.420892 15059 leader_election.cc:304] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:31.421001 15294 consensus_queue.cc:237] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.421164 15296 raft_consensus.cc:2802] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:31.421293 15296 raft_consensus.cc:491] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:31.421169 15312 raft_consensus.cc:2802] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:31.421399 15296 raft_consensus.cc:3058] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.421481 15312 raft_consensus.cc:695] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 LEADER]: Becoming Leader. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Running, Role: LEADER
I20250905 07:50:31.421738 15312 consensus_queue.cc:237] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.424868 15296 raft_consensus.cc:513] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.425606 15244 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40" is_pre_election: true
I20250905 07:50:31.425868 15244 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 0.
I20250905 07:50:31.425915 15296 leader_election.cc:290] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 1 election: Requested vote from peers 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935), 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:31.426152 15170 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7"
I20250905 07:50:31.426369 15170 raft_consensus.cc:3058] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.426684 15244 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40"
I20250905 07:50:31.426887 15244 raft_consensus.cc:3058] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.429297 15170 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 1.
I20250905 07:50:31.429662 15059 leader_election.cc:304] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:31.429934 15296 raft_consensus.cc:2802] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:31.430630 15244 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 1.
I20250905 07:50:31.433189 15094 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" is_pre_election: true
I20250905 07:50:31.434150 15244 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40" is_pre_election: true
I20250905 07:50:31.434350 15296 raft_consensus.cc:695] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 LEADER]: Becoming Leader. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Running, Role: LEADER
I20250905 07:50:31.434459 15244 raft_consensus.cc:2391] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96f1548c7bd14e87a78f184f668ce7f7 in current term 1: Already voted for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in this term.
I20250905 07:50:31.434715 15296 consensus_queue.cc:237] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:31.436717 14993 catalog_manager.cc:5582] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e1f7067faba44858d1628228fd94c40 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "9e1f7067faba44858d1628228fd94c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:31.437443 15135 leader_election.cc:304] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96f1548c7bd14e87a78f184f668ce7f7; no voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 9e1f7067faba44858d1628228fd94c40
I20250905 07:50:31.437726 15300 raft_consensus.cc:2747] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:50:31.437811 14992 catalog_manager.cc:5582] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65). New cstate: current_term: 1 leader_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:31.441049 14992 catalog_manager.cc:5582] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e1f7067faba44858d1628228fd94c40 (127.8.229.67). New cstate: current_term: 1 leader_uuid: "9e1f7067faba44858d1628228fd94c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:31.606540 15300 raft_consensus.cc:491] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:31.606743 15300 raft_consensus.cc:513] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.607402 15300 leader_election.cc:290] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381), 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883)
I20250905 07:50:31.607893 15094 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" is_pre_election: true
I20250905 07:50:31.607905 15244 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40" is_pre_election: true
I20250905 07:50:31.608096 15094 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 0.
I20250905 07:50:31.608145 15244 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 0.
I20250905 07:50:31.608469 15135 leader_election.cc:304] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:31.608697 15300 raft_consensus.cc:2802] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:31.608777 15300 raft_consensus.cc:491] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:31.608850 15300 raft_consensus.cc:3058] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.610675 15300 raft_consensus.cc:513] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.611160 15300 leader_election.cc:290] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 1 election: Requested vote from peers 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381), 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883)
I20250905 07:50:31.611510 15244 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40"
I20250905 07:50:31.611542 15094 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
I20250905 07:50:31.611687 15244 raft_consensus.cc:3058] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.611702 15094 raft_consensus.cc:3058] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:31.613723 15094 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 1.
I20250905 07:50:31.613869 15244 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 1.
I20250905 07:50:31.614106 15135 leader_election.cc:304] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:31.614392 15300 raft_consensus.cc:2802] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:31.614750 15300 raft_consensus.cc:695] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 LEADER]: Becoming Leader. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Running, Role: LEADER
I20250905 07:50:31.614993 15300 consensus_queue.cc:237] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:31.617187 14992 catalog_manager.cc:5582] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "96f1548c7bd14e87a78f184f668ce7f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:31.656128 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:50:31.670434 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:31.671021 9109 tablet_replica.cc:331] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: stopping tablet replica
I20250905 07:50:31.671284 9109 raft_consensus.cc:2241] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.671545 9109 raft_consensus.cc:2270] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.672446 9109 tablet_replica.cc:331] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: stopping tablet replica
I20250905 07:50:31.672642 9109 raft_consensus.cc:2241] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.672804 9109 raft_consensus.cc:2270] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.673508 9109 tablet_replica.cc:331] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: stopping tablet replica
I20250905 07:50:31.673693 9109 raft_consensus.cc:2241] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.673858 9109 raft_consensus.cc:2270] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.674436 9109 tablet_replica.cc:331] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: stopping tablet replica
I20250905 07:50:31.674613 9109 raft_consensus.cc:2241] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:31.674892 9109 pending_rounds.cc:70] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Trying to abort 1 pending ops.
I20250905 07:50:31.674966 9109 pending_rounds.cc:77] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912154360135680 op_type: NO_OP noop_request { }
I20250905 07:50:31.675113 9109 raft_consensus.cc:2887] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:31.675267 9109 raft_consensus.cc:2270] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.691351 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
I20250905 07:50:31.695042 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:50:31.714556 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:31.714883 9109 tablet_replica.cc:331] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: stopping tablet replica
I20250905 07:50:31.715045 9109 raft_consensus.cc:2241] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.715205 9109 raft_consensus.cc:2270] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.715860 9109 tablet_replica.cc:331] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: stopping tablet replica
I20250905 07:50:31.716005 9109 raft_consensus.cc:2241] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:31.716195 9109 pending_rounds.cc:70] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Trying to abort 1 pending ops.
I20250905 07:50:31.716235 9109 pending_rounds.cc:77] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912155097006080 op_type: NO_OP noop_request { }
I20250905 07:50:31.716334 9109 raft_consensus.cc:2887] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:31.716419 9109 raft_consensus.cc:2270] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.717029 9109 tablet_replica.cc:331] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: stopping tablet replica
I20250905 07:50:31.717172 9109 raft_consensus.cc:2241] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.717303 9109 raft_consensus.cc:2270] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.717765 9109 tablet_replica.cc:331] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: stopping tablet replica
I20250905 07:50:31.717880 9109 raft_consensus.cc:2241] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.717986 9109 raft_consensus.cc:2270] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.732511 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:50:31.735906 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:50:31.744735 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:31.745060 9109 tablet_replica.cc:331] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: stopping tablet replica
I20250905 07:50:31.745275 9109 raft_consensus.cc:2241] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.745422 9109 raft_consensus.cc:2270] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.746070 9109 tablet_replica.cc:331] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: stopping tablet replica
I20250905 07:50:31.746181 9109 raft_consensus.cc:2241] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:31.746337 9109 pending_rounds.cc:70] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Trying to abort 1 pending ops.
I20250905 07:50:31.746371 9109 pending_rounds.cc:77] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912154306007040 op_type: NO_OP noop_request { }
I20250905 07:50:31.746461 9109 raft_consensus.cc:2887] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:31.746521 9109 raft_consensus.cc:2270] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.747175 9109 tablet_replica.cc:331] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: stopping tablet replica
I20250905 07:50:31.747282 9109 raft_consensus.cc:2241] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:31.747375 9109 raft_consensus.cc:2270] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.747905 9109 tablet_replica.cc:331] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: stopping tablet replica
I20250905 07:50:31.748008 9109 raft_consensus.cc:2241] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:31.748188 9109 pending_rounds.cc:70] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Trying to abort 1 pending ops.
I20250905 07:50:31.748225 9109 pending_rounds.cc:77] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912154303840256 op_type: NO_OP noop_request { }
I20250905 07:50:31.748286 9109 raft_consensus.cc:2887] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:31.748344 9109 raft_consensus.cc:2270] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:31.753021 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:50:31.756608 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:31.758734 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:31.758914 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:31.759636 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:31.760331 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:31.760532 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:31.760599 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:31.760669 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058631760665 us; error 0 us; skew 500 ppm
I20250905 07:50:31.760816 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:31.761888 9109 webserver.cc:480] Webserver started at http://127.8.229.68:45853/ using document root <none> and password file <none>
I20250905 07:50:31.762108 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:31.762217 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:31.762347 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:31.762924 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2"
format_stamp: "Formatted at 2025-09-05 07:50:31 on dist-test-slave-pgzt"
I20250905 07:50:31.765352 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:50:31.766862 15330 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:31.767264 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250905 07:50:31.767376 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2"
format_stamp: "Formatted at 2025-09-05 07:50:31 on dist-test-slave-pgzt"
I20250905 07:50:31.767539 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:31.776930 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:31.777442 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:31.777889 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:31.778648 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:31.778720 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:31.778791 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:31.778829 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:31.791034 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:36753
I20250905 07:50:31.791059 15392 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:36753 every 8 connection(s)
I20250905 07:50:31.797619 15393 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39689
I20250905 07:50:31.797750 15393 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:31.798050 15393 heartbeater.cc:507] Master 127.8.229.126:39689 requested a full tablet report, sending...
I20250905 07:50:31.798842 14992 ts_manager.cc:194] Registered new tserver with Master: 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68:36753)
I20250905 07:50:31.799628 14992 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46478
W20250905 07:50:32.046383 15041 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:32.049437 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:32.049533 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:32.051075 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:32.051159 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:32.052716 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:32.052795 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
I20250905 07:50:32.800801 15393 heartbeater.cc:499] Master 127.8.229.126:39689 was elected leader, sending a full tablet report...
W20250905 07:50:33.056613 15041 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:33.058605 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:33.058687 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:33.060081 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:33.060163 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:33.061528 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111)
W20250905 07:50:33.061594 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111)
W20250905 07:50:34.065557 15041 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:34.066646 14977 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20250905 07:50:34.067672 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:34.067741 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:34.069151 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111)
W20250905 07:50:34.069217 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111)
W20250905 07:50:34.070564 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:34.070622 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:35.074316 15041 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111)
W20250905 07:50:35.076623 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111)
W20250905 07:50:35.076699 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111)
W20250905 07:50:35.078330 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:35.078406 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:35.080037 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:35.080122 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:36.083825 15041 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:36.085825 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:36.085908 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111)
W20250905 07:50:36.087234 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:36.087291 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:36.088791 15041 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
W20250905 07:50:36.088850 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.8.229.67:46381: connect: Connection refused (error 111)
I20250905 07:50:37.066547 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:37.069209 15412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:37.069331 15413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:37.069824 15415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:37.077409 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:37.078416 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:37.078480 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:37.078521 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058637078502 us; error 0 us; skew 500 ppm
I20250905 07:50:37.078701 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:37.080018 9109 webserver.cc:480] Webserver started at http://127.8.229.65:40519/ using document root <none> and password file <none>
I20250905 07:50:37.080247 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:37.080322 9109 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250905 07:50:37.082326 9109 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:37.083886 15420 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:37.084293 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:50:37.084404 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:37.084537 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20250905 07:50:37.089601 15041 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 9e1f7067faba44858d1628228fd94c40 not available for placement
I20250905 07:50:37.104146 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:37.104678 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:37.105217 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:37.106566 15427 ts_tablet_manager.cc:542] Loading tablet metadata (0/4 complete)
I20250905 07:50:37.114939 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20250905 07:50:37.115051 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.001s sys 0.000s
I20250905 07:50:37.115177 9109 ts_tablet_manager.cc:594] Registering tablets (0/4 complete)
I20250905 07:50:37.117267 15427 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:37.120935 15427 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:37.121340 15427 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap complete.
I20250905 07:50:37.121600 15427 ts_tablet_manager.cc:1397] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:50:37.122220 15427 raft_consensus.cc:357] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:37.122383 15427 raft_consensus.cc:738] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.122591 9109 ts_tablet_manager.cc:610] Registered 4 tablets
I20250905 07:50:37.122702 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.008s user 0.002s sys 0.003s
I20250905 07:50:37.122623 15427 consensus_queue.cc:260] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:37.123389 15427 ts_tablet_manager.cc:1428] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:37.123696 15427 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:37.142602 15485 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:41883 every 8 connection(s)
I20250905 07:50:37.142612 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:41883
I20250905 07:50:37.144140 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:50:37.149793 15427 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20250905 07:50:37.150341 15492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:37.150367 15427 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap complete.
W20250905 07:50:37.150339 15491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:37.150840 15427 ts_tablet_manager.cc:1397] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.027s user 0.018s sys 0.008s
W20250905 07:50:37.151173 15494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:37.152009 15427 raft_consensus.cc:357] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.152271 15427 raft_consensus.cc:738] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.152542 15427 consensus_queue.cc:260] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.153146 15427 ts_tablet_manager.cc:1428] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:50:37.153448 15427 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:37.157012 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:37.157296 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:37.157377 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:37.157421 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058637157418 us; error 0 us; skew 500 ppm
I20250905 07:50:37.157521 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:37.158669 15427 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250905 07:50:37.158898 9109 webserver.cc:480] Webserver started at http://127.8.229.66:36569/ using document root <none> and password file <none>
I20250905 07:50:37.159152 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:37.159225 9109 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250905 07:50:37.159253 15427 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap complete.
I20250905 07:50:37.159536 15427 ts_tablet_manager.cc:1397] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.006s
I20250905 07:50:37.160120 15427 raft_consensus.cc:357] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.160358 15427 raft_consensus.cc:738] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.160579 15427 consensus_queue.cc:260] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.161139 15427 ts_tablet_manager.cc:1428] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:37.161152 9109 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:37.161393 15427 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap starting.
I20250905 07:50:37.162671 15500 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:37.163079 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:37.163178 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "96f1548c7bd14e87a78f184f668ce7f7"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:37.163275 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:37.165266 15427 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:37.165557 15427 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Bootstrap complete.
I20250905 07:50:37.165759 15427 ts_tablet_manager.cc:1397] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20250905 07:50:37.166229 15427 raft_consensus.cc:357] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.166353 15427 raft_consensus.cc:738] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.166550 15427 consensus_queue.cc:260] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.167479 15427 ts_tablet_manager.cc:1428] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:37.167632 15486 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39689
I20250905 07:50:37.167748 15486 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:37.168108 15486 heartbeater.cc:507] Master 127.8.229.126:39689 requested a full tablet report, sending...
I20250905 07:50:37.169330 14992 ts_manager.cc:194] Re-registered known tserver with Master: 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883)
I20250905 07:50:37.170919 14992 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46486
I20250905 07:50:37.172199 15486 heartbeater.cc:499] Master 127.8.229.126:39689 was elected leader, sending a full tablet report...
I20250905 07:50:37.172968 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:37.173472 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:37.174093 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:37.182693 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20250905 07:50:37.182798 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20250905 07:50:37.184712 15508 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:37.189044 15508 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250905 07:50:37.189245 9109 ts_tablet_manager.cc:610] Registered 4 tablets
I20250905 07:50:37.189316 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.006s user 0.005s sys 0.000s
I20250905 07:50:37.189461 15508 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap complete.
I20250905 07:50:37.189736 15508 ts_tablet_manager.cc:1397] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250905 07:50:37.190351 15508 raft_consensus.cc:357] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:37.190544 15508 raft_consensus.cc:738] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.190788 15508 consensus_queue.cc:260] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:37.191531 15508 ts_tablet_manager.cc:1428] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:37.191864 15508 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:37.196448 15508 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:37.196744 15508 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap complete.
I20250905 07:50:37.197016 15508 ts_tablet_manager.cc:1397] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20250905 07:50:37.197657 15508 raft_consensus.cc:357] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.197790 15508 raft_consensus.cc:738] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.198041 15508 consensus_queue.cc:260] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.198513 15508 ts_tablet_manager.cc:1428] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:37.198701 15508 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:37.203517 15508 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:37.203941 15508 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap complete.
I20250905 07:50:37.204188 15508 ts_tablet_manager.cc:1397] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20250905 07:50:37.204808 15508 raft_consensus.cc:357] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.205011 15508 raft_consensus.cc:738] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.205250 15508 consensus_queue.cc:260] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.205615 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:36935
I20250905 07:50:37.205657 15568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:36935 every 8 connection(s)
I20250905 07:50:37.205880 15508 ts_tablet_manager.cc:1428] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:37.206084 15508 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap starting.
I20250905 07:50:37.212973 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:50:37.214713 15508 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:37.215037 15508 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Bootstrap complete.
I20250905 07:50:37.215262 15508 ts_tablet_manager.cc:1397] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20250905 07:50:37.216008 15508 raft_consensus.cc:357] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.216269 15508 raft_consensus.cc:738] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.216565 15508 consensus_queue.cc:260] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
W20250905 07:50:37.217247 15573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:37.217468 15508 ts_tablet_manager.cc:1428] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20250905 07:50:37.218533 15574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:37.218945 15569 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39689
I20250905 07:50:37.219099 15569 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:37.219516 15569 heartbeater.cc:507] Master 127.8.229.126:39689 requested a full tablet report, sending...
I20250905 07:50:37.220042 9109 server_base.cc:1047] running on GCE node
W20250905 07:50:37.220240 15576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:37.220629 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:37.220703 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:37.220736 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058637220733 us; error 0 us; skew 500 ppm
I20250905 07:50:37.220887 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:37.221210 14992 ts_manager.cc:194] Re-registered known tserver with Master: 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:37.222110 9109 webserver.cc:480] Webserver started at http://127.8.229.67:35667/ using document root <none> and password file <none>
I20250905 07:50:37.222296 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:37.222339 9109 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250905 07:50:37.223224 14992 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46494
I20250905 07:50:37.224135 9109 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:50:37.224567 15569 heartbeater.cc:499] Master 127.8.229.126:39689 was elected leader, sending a full tablet report...
I20250905 07:50:37.225879 15581 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:37.226230 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:50:37.226330 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "9e1f7067faba44858d1628228fd94c40"
format_stamp: "Formatted at 2025-09-05 07:50:30 on dist-test-slave-pgzt"
I20250905 07:50:37.226462 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:37.253676 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:37.254285 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:37.254875 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:37.263335 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20250905 07:50:37.263470 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.001s
I20250905 07:50:37.265662 15588 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:37.269824 15588 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:37.270128 15588 tablet_bootstrap.cc:492] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Bootstrap complete.
I20250905 07:50:37.270350 15588 ts_tablet_manager.cc:1397] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250905 07:50:37.270949 15588 raft_consensus.cc:357] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:37.271102 15588 raft_consensus.cc:738] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.271317 15588 consensus_queue.cc:260] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:37.271775 9109 ts_tablet_manager.cc:610] Registered 4 tablets
I20250905 07:50:37.271842 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.008s user 0.007s sys 0.001s
I20250905 07:50:37.272083 15588 ts_tablet_manager.cc:1428] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:50:37.272369 15588 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:37.277257 15588 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250905 07:50:37.277712 15588 tablet_bootstrap.cc:492] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Bootstrap complete.
I20250905 07:50:37.277933 15588 ts_tablet_manager.cc:1397] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20250905 07:50:37.278553 15588 raft_consensus.cc:357] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.278766 15588 raft_consensus.cc:738] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.279031 15588 consensus_queue.cc:260] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.279542 15588 ts_tablet_manager.cc:1428] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20250905 07:50:37.279727 15588 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:37.285086 15588 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:37.285501 15588 tablet_bootstrap.cc:492] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Bootstrap complete.
I20250905 07:50:37.285797 15588 ts_tablet_manager.cc:1397] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20250905 07:50:37.286705 15588 raft_consensus.cc:357] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.286939 15588 raft_consensus.cc:738] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.287321 15588 consensus_queue.cc:260] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.288005 15588 ts_tablet_manager.cc:1428] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:37.288276 15588 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Bootstrap starting.
I20250905 07:50:37.289289 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:46381
I20250905 07:50:37.289361 15648 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:46381 every 8 connection(s)
I20250905 07:50:37.294384 15588 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250905 07:50:37.294891 15588 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Bootstrap complete.
I20250905 07:50:37.295199 15588 ts_tablet_manager.cc:1397] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250905 07:50:37.296247 15588 raft_consensus.cc:357] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.296661 15588 raft_consensus.cc:738] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Initialized, Role: FOLLOWER
I20250905 07:50:37.297008 15588 consensus_queue.cc:260] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:37.297605 15588 ts_tablet_manager.cc:1428] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:37.298143 15649 heartbeater.cc:344] Connected to a master server at 127.8.229.126:39689
I20250905 07:50:37.298296 15649 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:37.298652 15649 heartbeater.cc:507] Master 127.8.229.126:39689 requested a full tablet report, sending...
I20250905 07:50:37.299944 14992 ts_manager.cc:194] Re-registered known tserver with Master: 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:37.301630 14992 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46508
I20250905 07:50:37.302884 15649 heartbeater.cc:499] Master 127.8.229.126:39689 was elected leader, sending a full tablet report...
W20250905 07:50:38.098819 15041 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Replica 9e1f7067faba44858d1628228fd94c40 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } }
W20250905 07:50:38.100401 15041 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 306950a7fe3940348981685598068091, TS 96f1548c7bd14e87a78f184f668ce7f7 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
W20250905 07:50:38.100509 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 306950a7fe3940348981685598068091, TS 96f1548c7bd14e87a78f184f668ce7f7 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
W20250905 07:50:38.106715 15041 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet bebc2b83c54040d9a8dccb86347c42c8, TS 9e1f7067faba44858d1628228fd94c40 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
W20250905 07:50:38.106823 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet bebc2b83c54040d9a8dccb86347c42c8, TS 9e1f7067faba44858d1628228fd94c40 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
W20250905 07:50:38.112843 15041 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 46bbd12bea1345e888aae6cebac949b4, TS 4566e16eabc044a1a3ce7ad386b6a3f3 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
W20250905 07:50:38.113024 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 46bbd12bea1345e888aae6cebac949b4, TS 4566e16eabc044a1a3ce7ad386b6a3f3 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:38.390560 15657 raft_consensus.cc:491] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:38.390779 15657 raft_consensus.cc:513] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.391633 15657 leader_election.cc:290] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935), 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:38.397234 15623 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40" is_pre_election: true
I20250905 07:50:38.397261 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7" is_pre_election: true
I20250905 07:50:38.397435 15623 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 1.
I20250905 07:50:38.397467 15543 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 1.
I20250905 07:50:38.397840 15422 leader_election.cc:304] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:38.398051 15657 raft_consensus.cc:2802] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:38.398120 15657 raft_consensus.cc:491] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:38.398204 15657 raft_consensus.cc:3058] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.400156 15657 raft_consensus.cc:513] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.400632 15657 leader_election.cc:290] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 election: Requested vote from peers 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935), 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:38.401000 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7"
I20250905 07:50:38.401085 15623 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bebc2b83c54040d9a8dccb86347c42c8" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40"
I20250905 07:50:38.401132 15543 raft_consensus.cc:3058] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.401269 15623 raft_consensus.cc:3058] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.403158 15543 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 2.
I20250905 07:50:38.403265 15623 raft_consensus.cc:2466] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 2.
I20250905 07:50:38.403493 15422 leader_election.cc:304] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:38.403717 15657 raft_consensus.cc:2802] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:38.404059 15657 raft_consensus.cc:695] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 LEADER]: Becoming Leader. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Running, Role: LEADER
I20250905 07:50:38.404281 15657 consensus_queue.cc:237] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.406740 14992 catalog_manager.cc:5582] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:38.503479 15666 raft_consensus.cc:491] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:38.503703 15666 raft_consensus.cc:513] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:38.504588 15666 leader_election.cc:290] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381), 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883)
I20250905 07:50:38.510308 15461 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" is_pre_election: true
I20250905 07:50:38.510329 15623 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40" is_pre_election: true
I20250905 07:50:38.510517 15461 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 1.
I20250905 07:50:38.510555 15623 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 1.
I20250905 07:50:38.510870 15502 leader_election.cc:304] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96f1548c7bd14e87a78f184f668ce7f7, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:38.511116 15666 raft_consensus.cc:2802] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:38.511189 15666 raft_consensus.cc:491] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:38.511267 15666 raft_consensus.cc:3058] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.513303 15666 raft_consensus.cc:513] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:38.513823 15666 leader_election.cc:290] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 2 election: Requested vote from peers 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381), 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883)
I20250905 07:50:38.514218 15623 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40"
I20250905 07:50:38.514220 15461 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "46bbd12bea1345e888aae6cebac949b4" candidate_uuid: "96f1548c7bd14e87a78f184f668ce7f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
I20250905 07:50:38.514390 15623 raft_consensus.cc:3058] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.514405 15461 raft_consensus.cc:3058] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.516389 15623 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 2.
I20250905 07:50:38.516391 15461 raft_consensus.cc:2466] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f1548c7bd14e87a78f184f668ce7f7 in term 2.
I20250905 07:50:38.516742 15504 leader_election.cc:304] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:38.516963 15666 raft_consensus.cc:2802] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:38.517319 15666 raft_consensus.cc:695] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 LEADER]: Becoming Leader. State: Replica: 96f1548c7bd14e87a78f184f668ce7f7, State: Running, Role: LEADER
I20250905 07:50:38.517540 15666 consensus_queue.cc:237] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:38.520071 14992 catalog_manager.cc:5582] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "96f1548c7bd14e87a78f184f668ce7f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:38.544972 15657 raft_consensus.cc:491] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:38.545166 15657 raft_consensus.cc:513] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.545780 15657 leader_election.cc:290] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935), 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:38.546273 15623 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40" is_pre_election: true
I20250905 07:50:38.546278 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7" is_pre_election: true
I20250905 07:50:38.546492 15543 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 1.
I20250905 07:50:38.546530 15623 raft_consensus.cc:2408] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4566e16eabc044a1a3ce7ad386b6a3f3 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250905 07:50:38.546825 15422 leader_election.cc:304] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:38.547060 15657 raft_consensus.cc:2802] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:38.547158 15657 raft_consensus.cc:491] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:38.547233 15657 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.549441 15657 raft_consensus.cc:513] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.549911 15657 leader_election.cc:290] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 election: Requested vote from peers 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935), 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:38.550310 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7"
I20250905 07:50:38.550310 15623 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e1f7067faba44858d1628228fd94c40"
I20250905 07:50:38.550520 15623 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.550520 15543 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.553386 15543 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4566e16eabc044a1a3ce7ad386b6a3f3 in term 2.
I20250905 07:50:38.553467 15623 raft_consensus.cc:2408] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4566e16eabc044a1a3ce7ad386b6a3f3 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250905 07:50:38.553761 15422 leader_election.cc:304] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7; no voters:
I20250905 07:50:38.554018 15657 raft_consensus.cc:2802] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:38.554138 15657 raft_consensus.cc:695] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 LEADER]: Becoming Leader. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Running, Role: LEADER
I20250905 07:50:38.554383 15657 consensus_queue.cc:237] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.556743 14992 catalog_manager.cc:5582] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 reported cstate change: term changed from 1 to 2, leader changed from 9e1f7067faba44858d1628228fd94c40 (127.8.229.67) to 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65). New cstate: current_term: 2 leader_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:38.632205 15672 raft_consensus.cc:491] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:38.632397 15672 raft_consensus.cc:513] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.633150 15672 leader_election.cc:290] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:38.640794 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7" is_pre_election: true
I20250905 07:50:38.640811 15461 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" is_pre_election: true
I20250905 07:50:38.641019 15461 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 1.
I20250905 07:50:38.641001 15543 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 1.
I20250905 07:50:38.641436 15584 leader_election.cc:304] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 96f1548c7bd14e87a78f184f668ce7f7, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:38.641678 15672 raft_consensus.cc:2802] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:38.641757 15672 raft_consensus.cc:491] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:38.641832 15672 raft_consensus.cc:3058] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.644050 15672 raft_consensus.cc:513] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.644631 15672 leader_election.cc:290] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 2 election: Requested vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:38.645013 15461 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
I20250905 07:50:38.645043 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "33e2555117254d15ba710c94aa648eec" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7"
I20250905 07:50:38.645232 15543 raft_consensus.cc:3058] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.645237 15461 raft_consensus.cc:3058] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:38.648120 15461 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 2.
I20250905 07:50:38.648120 15543 raft_consensus.cc:2466] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 2.
I20250905 07:50:38.648522 15584 leader_election.cc:304] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4566e16eabc044a1a3ce7ad386b6a3f3, 9e1f7067faba44858d1628228fd94c40; no voters:
I20250905 07:50:38.648792 15672 raft_consensus.cc:2802] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:38.649222 15672 raft_consensus.cc:695] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 LEADER]: Becoming Leader. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Running, Role: LEADER
I20250905 07:50:38.649513 15672 consensus_queue.cc:237] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:38.651867 14992 catalog_manager.cc:5582] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "9e1f7067faba44858d1628228fd94c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:38.978440 15623 raft_consensus.cc:1273] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Refusing update from remote peer 96f1548c7bd14e87a78f184f668ce7f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:38.979018 15666 consensus_queue.cc:1035] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:38.983389 15461 raft_consensus.cc:1273] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Refusing update from remote peer 96f1548c7bd14e87a78f184f668ce7f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:38.984284 15666 consensus_queue.cc:1035] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:38.986737 15543 raft_consensus.cc:1273] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Refusing update from remote peer 4566e16eabc044a1a3ce7ad386b6a3f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:38.987406 15657 consensus_queue.cc:1035] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:38.991271 15623 raft_consensus.cc:1273] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Refusing update from remote peer 4566e16eabc044a1a3ce7ad386b6a3f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:38.991803 15657 consensus_queue.cc:1035] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:39.023304 15685 raft_consensus.cc:491] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:39.023552 15685 raft_consensus.cc:513] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:39.024235 15685 leader_election.cc:290] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:39.024729 15461 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" is_pre_election: true
I20250905 07:50:39.024736 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7" is_pre_election: true
I20250905 07:50:39.024966 15543 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 2.
I20250905 07:50:39.025338 15583 leader_election.cc:304] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96f1548c7bd14e87a78f184f668ce7f7, 9e1f7067faba44858d1628228fd94c40; no voters: 4566e16eabc044a1a3ce7ad386b6a3f3
I20250905 07:50:39.025596 15685 raft_consensus.cc:2802] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250905 07:50:39.025684 15685 raft_consensus.cc:491] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:39.025774 15685 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:50:39.027865 15685 raft_consensus.cc:513] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:39.028479 15685 leader_election.cc:290] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 3 election: Requested vote from peers 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883), 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935)
I20250905 07:50:39.028807 15461 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3"
I20250905 07:50:39.028837 15543 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "306950a7fe3940348981685598068091" candidate_uuid: "9e1f7067faba44858d1628228fd94c40" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96f1548c7bd14e87a78f184f668ce7f7"
I20250905 07:50:39.028977 15543 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:50:39.030820 15543 raft_consensus.cc:2466] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1f7067faba44858d1628228fd94c40 in term 3.
I20250905 07:50:39.031158 15583 leader_election.cc:304] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96f1548c7bd14e87a78f184f668ce7f7, 9e1f7067faba44858d1628228fd94c40; no voters: 4566e16eabc044a1a3ce7ad386b6a3f3
I20250905 07:50:39.031493 15685 raft_consensus.cc:2802] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 3 FOLLOWER]: Leader election won for term 3
I20250905 07:50:39.031649 15685 raft_consensus.cc:695] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 3 LEADER]: Becoming Leader. State: Replica: 9e1f7067faba44858d1628228fd94c40, State: Running, Role: LEADER
I20250905 07:50:39.031908 15685 consensus_queue.cc:237] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:39.034116 14993 catalog_manager.cc:5582] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 reported cstate change: term changed from 2 to 3, leader changed from 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65) to 9e1f7067faba44858d1628228fd94c40 (127.8.229.67). New cstate: current_term: 3 leader_uuid: "9e1f7067faba44858d1628228fd94c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:39.074287 15543 raft_consensus.cc:1238] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 3 FOLLOWER]: Rejecting Update request from peer 4566e16eabc044a1a3ce7ad386b6a3f3 for earlier term 2. Current term is 3. Ops: []
I20250905 07:50:39.074847 15660 consensus_queue.cc:1046] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:39.075140 15660 raft_consensus.cc:3053] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 LEADER]: Stepping down as leader of term 2
I20250905 07:50:39.075213 15660 raft_consensus.cc:738] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4566e16eabc044a1a3ce7ad386b6a3f3, State: Running, Role: LEADER
I20250905 07:50:39.075512 15660 consensus_queue.cc:260] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
I20250905 07:50:39.075821 15660 raft_consensus.cc:3058] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Advancing to term 3
I20250905 07:50:39.112561 15461 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:39.113193 15685 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:39.117285 15543 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:39.117700 15623 consensus_queue.cc:237] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: true } }
I20250905 07:50:39.120821 15460 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:39.121780 15543 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:39.122038 15675 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:39.122264 15685 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:39.125710 15675 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: true } } }
I20250905 07:50:39.126173 15543 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: true } } }
W20250905 07:50:39.129024 15583 consensus_peers.cc:489] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 -> Peer 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68:36753): Couldn't send request to peer 0bb8a29342f8448e95e40dd1ed6cbbb2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33e2555117254d15ba710c94aa648eec. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:39.129138 15461 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: true } } }
I20250905 07:50:39.131356 14993 catalog_manager.cc:5582] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 reported cstate change: config changed from index -1 to 3, NON_VOTER 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "9e1f7067faba44858d1628228fd94c40" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250905 07:50:39.132285 15041 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20250905 07:50:39.136056 15041 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 306950a7fe3940348981685598068091, TS 4566e16eabc044a1a3ce7ad386b6a3f3 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
W20250905 07:50:39.136171 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 306950a7fe3940348981685598068091, TS 4566e16eabc044a1a3ce7ad386b6a3f3 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } }
W20250905 07:50:39.138672 15041 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 46bbd12bea1345e888aae6cebac949b4, TS 9e1f7067faba44858d1628228fd94c40 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
W20250905 07:50:39.138744 15041 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 46bbd12bea1345e888aae6cebac949b4, TS 9e1f7067faba44858d1628228fd94c40 -> TS 0bb8a29342f8448e95e40dd1ed6cbbb2 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } } peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } }
I20250905 07:50:39.479481 15543 raft_consensus.cc:1273] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 3 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250905 07:50:39.480068 15688 consensus_queue.cc:1035] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:39.484336 15461 raft_consensus.cc:1273] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 3 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250905 07:50:39.484920 15685 consensus_queue.cc:1035] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:39.485177 15685 consensus_queue.cc:1230] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Peer 4566e16eabc044a1a3ce7ad386b6a3f3 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20250905 07:50:39.486197 15461 pending_rounds.cc:85] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Aborting all ops after (but not including) 0
I20250905 07:50:39.486316 15461 pending_rounds.cc:107] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20250905 07:50:39.486413 15461 raft_consensus.cc:2887] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250905 07:50:39.586470 15692 ts_tablet_manager.cc:927] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: Initiating tablet copy from peer 9e1f7067faba44858d1628228fd94c40 (127.8.229.67:46381)
I20250905 07:50:39.587041 15692 tablet_copy_client.cc:323] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:46381
I20250905 07:50:39.592939 15634 tablet_copy_service.cc:140] P 9e1f7067faba44858d1628228fd94c40: Received BeginTabletCopySession request for tablet 33e2555117254d15ba710c94aa648eec from peer 0bb8a29342f8448e95e40dd1ed6cbbb2 ({username='slave'} at 127.0.0.1:45508)
I20250905 07:50:39.593098 15634 tablet_copy_service.cc:161] P 9e1f7067faba44858d1628228fd94c40: Beginning new tablet copy session on tablet 33e2555117254d15ba710c94aa648eec from peer 0bb8a29342f8448e95e40dd1ed6cbbb2 at {username='slave'} at 127.0.0.1:45508: session id = 0bb8a29342f8448e95e40dd1ed6cbbb2-33e2555117254d15ba710c94aa648eec
I20250905 07:50:39.595772 15634 tablet_copy_source_session.cc:215] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:39.597170 15692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e2555117254d15ba710c94aa648eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:39.602926 15692 tablet_copy_client.cc:806] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:39.603161 15692 tablet_copy_client.cc:670] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:39.604593 15692 tablet_copy_client.cc:538] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:39.607148 15692 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: Bootstrap starting.
I20250905 07:50:39.612476 15692 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:39.612758 15692 tablet_bootstrap.cc:492] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: Bootstrap complete.
I20250905 07:50:39.613034 15692 ts_tablet_manager.cc:1397] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: Time spent bootstrapping tablet: real 0.006s user 0.008s sys 0.000s
I20250905 07:50:39.613685 15692 raft_consensus.cc:357] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: true } }
I20250905 07:50:39.613840 15692 raft_consensus.cc:738] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0bb8a29342f8448e95e40dd1ed6cbbb2, State: Initialized, Role: LEARNER
I20250905 07:50:39.614028 15692 consensus_queue.cc:260] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: true } }
I20250905 07:50:39.614699 15692 ts_tablet_manager.cc:1428] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:39.615243 15634 tablet_copy_service.cc:342] P 9e1f7067faba44858d1628228fd94c40: Request end of tablet copy session 0bb8a29342f8448e95e40dd1ed6cbbb2-33e2555117254d15ba710c94aa648eec received from {username='slave'} at 127.0.0.1:45508
I20250905 07:50:39.615355 15634 tablet_copy_service.cc:434] P 9e1f7067faba44858d1628228fd94c40: ending tablet copy session 0bb8a29342f8448e95e40dd1ed6cbbb2-33e2555117254d15ba710c94aa648eec on tablet 33e2555117254d15ba710c94aa648eec with peer 0bb8a29342f8448e95e40dd1ed6cbbb2
I20250905 07:50:40.070966 15368 raft_consensus.cc:1215] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250905 07:50:40.679065 15701 raft_consensus.cc:1062] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40: attempting to promote NON_VOTER 0bb8a29342f8448e95e40dd1ed6cbbb2 to VOTER
I20250905 07:50:40.679917 15701 consensus_queue.cc:237] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } }
I20250905 07:50:40.681882 15368 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 LEARNER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:40.682078 15543 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:40.682483 15701 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:40.682825 15461 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:40.682830 15702 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:40.683385 15704 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:40.686861 15702 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.687479 15543 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.687713 15461 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.691130 15368 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.691583 14991 catalog_manager.cc:5582] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 reported cstate change: config changed from index 3 to 4, 0bb8a29342f8448e95e40dd1ed6cbbb2 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9e1f7067faba44858d1628228fd94c40" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "96f1548c7bd14e87a78f184f668ce7f7" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 36935 } attrs { replace: true } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.703153 15623 consensus_queue.cc:237] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } }
I20250905 07:50:40.704897 15461 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:40.705022 15368 raft_consensus.cc:1273] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 FOLLOWER]: Refusing update from remote peer 9e1f7067faba44858d1628228fd94c40: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:40.705322 15704 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250905 07:50:40.705964 15704 consensus_queue.cc:1035] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250905 07:50:40.707296 15701 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.708918 15368 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.710263 15036 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 33e2555117254d15ba710c94aa648eec with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:40.710690 15461 raft_consensus.cc:2953] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } } }
I20250905 07:50:40.711980 14992 catalog_manager.cc:5582] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 reported cstate change: config changed from index 4 to 5, VOTER 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66) evicted. New cstate: current_term: 2 leader_uuid: "9e1f7067faba44858d1628228fd94c40" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4566e16eabc044a1a3ce7ad386b6a3f3" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e1f7067faba44858d1628228fd94c40" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 46381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bb8a29342f8448e95e40dd1ed6cbbb2" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 36753 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:40.721122 15533 tablet_service.cc:1515] Processing DeleteTablet for tablet 33e2555117254d15ba710c94aa648eec with delete_type TABLET_DATA_TOMBSTONED (TS 96f1548c7bd14e87a78f184f668ce7f7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51120
I20250905 07:50:40.721644 15716 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:40.721797 15716 raft_consensus.cc:2241] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:40.721972 15716 raft_consensus.cc:2270] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:40.722594 15716 ts_tablet_manager.cc:1905] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:50:40.727993 15716 ts_tablet_manager.cc:1918] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250905 07:50:40.728103 15716 log.cc:1199] T 33e2555117254d15ba710c94aa648eec P 96f1548c7bd14e87a78f184f668ce7f7: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/33e2555117254d15ba710c94aa648eec
I20250905 07:50:40.728754 14978 catalog_manager.cc:4928] TS 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935): tablet 33e2555117254d15ba710c94aa648eec (table test-workload [id=721383af9e624a91b8018ce5fe8646c8]) successfully deleted
I20250905 07:50:41.318215 9109 tablet_server.cc:178] TabletServer@127.8.229.65:41883 shutting down...
I20250905 07:50:41.326268 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:41.326530 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.326732 9109 raft_consensus.cc:2241] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.326980 9109 raft_consensus.cc:2270] T 33e2555117254d15ba710c94aa648eec P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.327859 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.328009 9109 raft_consensus.cc:2241] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:41.328294 9109 raft_consensus.cc:2270] T bebc2b83c54040d9a8dccb86347c42c8 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.328713 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.328812 9109 raft_consensus.cc:2241] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.328914 9109 raft_consensus.cc:2270] T 46bbd12bea1345e888aae6cebac949b4 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.329295 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.329387 9109 raft_consensus.cc:2241] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.329504 9109 raft_consensus.cc:2270] T 306950a7fe3940348981685598068091 P 4566e16eabc044a1a3ce7ad386b6a3f3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20250905 07:50:41.340133 15504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20250905 07:50:41.341210 15504 consensus_peers.cc:489] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 -> Peer 4566e16eabc044a1a3ce7ad386b6a3f3 (127.8.229.65:41883): Couldn't send request to peer 4566e16eabc044a1a3ce7ad386b6a3f3. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:41883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:41.343535 9109 tablet_server.cc:195] TabletServer@127.8.229.65:41883 shutdown complete.
I20250905 07:50:41.346889 9109 tablet_server.cc:178] TabletServer@127.8.229.66:36935 shutting down...
I20250905 07:50:41.355535 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:41.355764 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.355930 9109 raft_consensus.cc:2241] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.356099 9109 raft_consensus.cc:2270] T bebc2b83c54040d9a8dccb86347c42c8 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.356666 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.356798 9109 raft_consensus.cc:2241] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:41.357061 9109 raft_consensus.cc:2270] T 46bbd12bea1345e888aae6cebac949b4 P 96f1548c7bd14e87a78f184f668ce7f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.357494 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.357616 9109 raft_consensus.cc:2241] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.357743 9109 raft_consensus.cc:2270] T 306950a7fe3940348981685598068091 P 96f1548c7bd14e87a78f184f668ce7f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.372040 9109 tablet_server.cc:195] TabletServer@127.8.229.66:36935 shutdown complete.
I20250905 07:50:41.375326 9109 tablet_server.cc:178] TabletServer@127.8.229.67:46381 shutting down...
I20250905 07:50:41.384241 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
W20250905 07:50:41.384218 15583 consensus_peers.cc:489] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 -> Peer 96f1548c7bd14e87a78f184f668ce7f7 (127.8.229.66:36935): Couldn't send request to peer 96f1548c7bd14e87a78f184f668ce7f7. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.66:36935: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:41.384488 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.384703 9109 raft_consensus.cc:2241] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:41.384982 9109 raft_consensus.cc:2270] T 33e2555117254d15ba710c94aa648eec P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.385671 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.385808 9109 raft_consensus.cc:2241] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.385941 9109 raft_consensus.cc:2270] T bebc2b83c54040d9a8dccb86347c42c8 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.386400 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.386526 9109 raft_consensus.cc:2241] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.386653 9109 raft_consensus.cc:2270] T 46bbd12bea1345e888aae6cebac949b4 P 9e1f7067faba44858d1628228fd94c40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.387059 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:41.387187 9109 raft_consensus.cc:2241] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 3 LEADER]: Raft consensus shutting down.
I20250905 07:50:41.387423 9109 raft_consensus.cc:2270] T 306950a7fe3940348981685598068091 P 9e1f7067faba44858d1628228fd94c40 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.401818 9109 tablet_server.cc:195] TabletServer@127.8.229.67:46381 shutdown complete.
I20250905 07:50:41.405238 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:50:41.414479 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:41.414672 9109 tablet_replica.cc:331] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2: stopping tablet replica
I20250905 07:50:41.414836 9109 raft_consensus.cc:2241] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:41.415058 9109 raft_consensus.cc:2270] T 33e2555117254d15ba710c94aa648eec P 0bb8a29342f8448e95e40dd1ed6cbbb2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.429211 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:50:41.432221 9109 master.cc:561] Master@127.8.229.126:39689 shutting down...
I20250905 07:50:41.445708 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:41.445931 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:41.446035 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P 4be40101869c4960b9809c639b9d0252: stopping tablet replica
I20250905 07:50:41.459470 9109 master.cc:583] Master@127.8.229.126:39689 shutdown complete.
[ OK ] AutoRebalancerTest.TestHandlingFailedTservers (11471 ms)
[ RUN ] AutoRebalancerTest.TestDeletedTables
I20250905 07:50:41.473048 9109 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.229.126:34109
I20250905 07:50:41.473524 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:41.475616 15721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.475862 15722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.476307 15724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:41.476600 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:41.476805 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:41.476857 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:41.476902 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058641476897 us; error 0 us; skew 500 ppm
I20250905 07:50:41.477042 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:41.478157 9109 webserver.cc:480] Webserver started at http://127.8.229.126:38425/ using document root <none> and password file <none>
I20250905 07:50:41.478364 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:41.478425 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:41.478552 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:41.479130 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/master-0-root/instance:
uuid: "d364292a1f1f4200a3de95c3d8ff2088"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.481567 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:41.483011 15729 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.483511 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20250905 07:50:41.483626 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/master-0-root
uuid: "d364292a1f1f4200a3de95c3d8ff2088"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.483726 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:41.495672 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:41.496165 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:41.506947 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.126:34109
I20250905 07:50:41.506959 15780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.126:34109 every 8 connection(s)
I20250905 07:50:41.508212 15781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:41.512238 15781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: Bootstrap starting.
I20250905 07:50:41.513294 15781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:41.514642 15781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: No bootstrap required, opened a new log
I20250905 07:50:41.515142 15781 raft_consensus.cc:357] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d364292a1f1f4200a3de95c3d8ff2088" member_type: VOTER }
I20250905 07:50:41.515260 15781 raft_consensus.cc:383] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:41.515301 15781 raft_consensus.cc:738] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d364292a1f1f4200a3de95c3d8ff2088, State: Initialized, Role: FOLLOWER
I20250905 07:50:41.515487 15781 consensus_queue.cc:260] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d364292a1f1f4200a3de95c3d8ff2088" member_type: VOTER }
I20250905 07:50:41.515597 15781 raft_consensus.cc:397] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250905 07:50:41.515635 15781 raft_consensus.cc:491] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250905 07:50:41.515681 15781 raft_consensus.cc:3058] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:41.517273 15781 raft_consensus.cc:513] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d364292a1f1f4200a3de95c3d8ff2088" member_type: VOTER }
I20250905 07:50:41.517426 15781 leader_election.cc:304] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d364292a1f1f4200a3de95c3d8ff2088; no voters:
I20250905 07:50:41.517729 15781 leader_election.cc:290] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250905 07:50:41.517830 15784 raft_consensus.cc:2802] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:41.518301 15784 raft_consensus.cc:695] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 1 LEADER]: Becoming Leader. State: Replica: d364292a1f1f4200a3de95c3d8ff2088, State: Running, Role: LEADER
I20250905 07:50:41.518473 15781 sys_catalog.cc:564] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [sys.catalog]: configured and running, proceeding with master startup.
I20250905 07:50:41.518488 15784 consensus_queue.cc:237] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d364292a1f1f4200a3de95c3d8ff2088" member_type: VOTER }
I20250905 07:50:41.519250 15786 sys_catalog.cc:455] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d364292a1f1f4200a3de95c3d8ff2088. Latest consensus state: current_term: 1 leader_uuid: "d364292a1f1f4200a3de95c3d8ff2088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d364292a1f1f4200a3de95c3d8ff2088" member_type: VOTER } }
I20250905 07:50:41.519464 15786 sys_catalog.cc:458] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:41.519502 15785 sys_catalog.cc:455] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d364292a1f1f4200a3de95c3d8ff2088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d364292a1f1f4200a3de95c3d8ff2088" member_type: VOTER } }
I20250905 07:50:41.519747 15785 sys_catalog.cc:458] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [sys.catalog]: This master's current role is: LEADER
I20250905 07:50:41.520769 15789 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250905 07:50:41.522007 15789 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250905 07:50:41.523623 9109 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250905 07:50:41.524813 15789 catalog_manager.cc:1349] Generated new cluster ID: 24720a4860ae4161945d878b36c40549
I20250905 07:50:41.524886 15789 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250905 07:50:41.550846 15789 catalog_manager.cc:1372] Generated new certificate authority record
I20250905 07:50:41.551643 15789 catalog_manager.cc:1506] Loading token signing keys...
I20250905 07:50:41.558218 15789 catalog_manager.cc:5955] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: Generated new TSK 0
I20250905 07:50:41.558452 15789 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250905 07:50:41.588537 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:41.591288 15802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.591282 15803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.591763 15805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:41.591935 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:41.592159 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:41.592216 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:41.592262 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058641592256 us; error 0 us; skew 500 ppm
I20250905 07:50:41.592409 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:41.593492 9109 webserver.cc:480] Webserver started at http://127.8.229.65:40795/ using document root <none> and password file <none>
I20250905 07:50:41.593713 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:41.593789 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:41.593919 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:41.594545 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/instance:
uuid: "8673871edcb144dc863da6e6fb876fad"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.597234 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250905 07:50:41.598615 15810 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.598907 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20250905 07:50:41.598994 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "8673871edcb144dc863da6e6fb876fad"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.599100 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:41.620096 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:41.620661 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:41.621245 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:41.622095 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:41.622150 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.622242 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:41.622279 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.633704 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:44917
I20250905 07:50:41.633731 15872 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:44917 every 8 connection(s)
I20250905 07:50:41.635293 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:41.637936 15877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.638088 15878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.639053 15880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:41.639231 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:41.639601 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:41.639696 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:41.639743 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058641639739 us; error 0 us; skew 500 ppm
I20250905 07:50:41.639886 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:41.641260 9109 webserver.cc:480] Webserver started at http://127.8.229.66:44043/ using document root <none> and password file <none>
I20250905 07:50:41.641492 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:41.641570 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:41.641670 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:41.642112 15873 heartbeater.cc:344] Connected to a master server at 127.8.229.126:34109
I20250905 07:50:41.642233 15873 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:41.642421 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root/instance:
uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.642552 15873 heartbeater.cc:507] Master 127.8.229.126:34109 requested a full tablet report, sending...
I20250905 07:50:41.643508 15746 ts_manager.cc:194] Registered new tserver with Master: 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917)
I20250905 07:50:41.644902 15746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39974
I20250905 07:50:41.645385 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250905 07:50:41.646814 15885 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.647167 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:41.647264 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root
uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.647373 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:41.666836 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:41.667330 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:41.667861 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:41.668648 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:41.668702 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.668778 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:41.668816 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.685269 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.66:46761
I20250905 07:50:41.685302 15947 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.66:46761 every 8 connection(s)
I20250905 07:50:41.686744 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:41.689509 15951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.689740 15952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:41.691071 15954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:41.691409 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:41.691673 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:41.691727 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:41.691752 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058641691749 us; error 0 us; skew 500 ppm
I20250905 07:50:41.691840 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:41.693051 9109 webserver.cc:480] Webserver started at http://127.8.229.67:42953/ using document root <none> and password file <none>
I20250905 07:50:41.693248 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:41.693328 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:41.693466 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:41.693468 15948 heartbeater.cc:344] Connected to a master server at 127.8.229.126:34109
I20250905 07:50:41.693625 15948 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:41.693919 15948 heartbeater.cc:507] Master 127.8.229.126:34109 requested a full tablet report, sending...
I20250905 07:50:41.694077 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root/instance:
uuid: "a6f256ebcfff460ea87eaa92563a9231"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.694689 15746 ts_manager.cc:194] Registered new tserver with Master: 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:41.695402 15746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39988
I20250905 07:50:41.696763 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:41.698024 15959 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.698330 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:41.698421 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root
uuid: "a6f256ebcfff460ea87eaa92563a9231"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:41.698513 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:41.712085 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:41.712519 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:41.713002 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:41.713794 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:41.713852 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.713932 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:41.713975 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:41.725265 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.67:37795
I20250905 07:50:41.725301 16021 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.67:37795 every 8 connection(s)
I20250905 07:50:41.732484 16022 heartbeater.cc:344] Connected to a master server at 127.8.229.126:34109
I20250905 07:50:41.732609 16022 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:41.732887 16022 heartbeater.cc:507] Master 127.8.229.126:34109 requested a full tablet report, sending...
I20250905 07:50:41.733639 15746 ts_manager.cc:194] Registered new tserver with Master: a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:41.733963 9109 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007914178s
I20250905 07:50:41.734407 15746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39994
I20250905 07:50:42.646142 15873 heartbeater.cc:499] Master 127.8.229.126:34109 was elected leader, sending a full tablet report...
I20250905 07:50:42.696671 15948 heartbeater.cc:499] Master 127.8.229.126:34109 was elected leader, sending a full tablet report...
I20250905 07:50:42.735656 16022 heartbeater.cc:499] Master 127.8.229.126:34109 was elected leader, sending a full tablet report...
I20250905 07:50:42.761335 9109 test_util.cc:276] Using random seed: 406869616
I20250905 07:50:42.778569 15746 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40004:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:50:42.779382 15746 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:50:42.826833 15838 tablet_service.cc:1468] Processing CreateTablet for tablet 2df7d42ec18145c1b84430ee2e02ed00 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:42.827001 15837 tablet_service.cc:1468] Processing CreateTablet for tablet 135d618a63ea48448bf95be21e340ce5 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:42.827414 15838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2df7d42ec18145c1b84430ee2e02ed00. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.827492 15836 tablet_service.cc:1468] Processing CreateTablet for tablet db5db38b839d4ff6b71ba9d9735fa73a (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:42.827737 15985 tablet_service.cc:1468] Processing CreateTablet for tablet db5db38b839d4ff6b71ba9d9735fa73a (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:42.828073 15985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db5db38b839d4ff6b71ba9d9735fa73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.827942 15835 tablet_service.cc:1468] Processing CreateTablet for tablet 2c894b6ee25a4fac8290c1edc8243572 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:42.830515 15984 tablet_service.cc:1468] Processing CreateTablet for tablet 2c894b6ee25a4fac8290c1edc8243572 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:42.830909 15984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c894b6ee25a4fac8290c1edc8243572. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.832753 15986 tablet_service.cc:1468] Processing CreateTablet for tablet 135d618a63ea48448bf95be21e340ce5 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:42.833194 15986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135d618a63ea48448bf95be21e340ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.833338 15913 tablet_service.cc:1468] Processing CreateTablet for tablet 2df7d42ec18145c1b84430ee2e02ed00 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:42.833674 15913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2df7d42ec18145c1b84430ee2e02ed00. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.835760 15987 tablet_service.cc:1468] Processing CreateTablet for tablet 2df7d42ec18145c1b84430ee2e02ed00 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:42.836115 15987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2df7d42ec18145c1b84430ee2e02ed00. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.838937 15911 tablet_service.cc:1468] Processing CreateTablet for tablet db5db38b839d4ff6b71ba9d9735fa73a (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:42.839390 15911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db5db38b839d4ff6b71ba9d9735fa73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.839886 15912 tablet_service.cc:1468] Processing CreateTablet for tablet 135d618a63ea48448bf95be21e340ce5 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:42.840420 15912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135d618a63ea48448bf95be21e340ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.843276 15910 tablet_service.cc:1468] Processing CreateTablet for tablet 2c894b6ee25a4fac8290c1edc8243572 (DEFAULT_TABLE table=test-workload [id=60f301b140254606998f8e2e5c8f4c51]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:42.843904 15910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c894b6ee25a4fac8290c1edc8243572. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.847949 15836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db5db38b839d4ff6b71ba9d9735fa73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.848774 16043 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:42.850484 16043 tablet_bootstrap.cc:654] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.852044 15835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c894b6ee25a4fac8290c1edc8243572. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.855969 15837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135d618a63ea48448bf95be21e340ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:42.858960 16043 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:42.859079 16043 ts_tablet_manager.cc:1397] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20250905 07:50:42.859872 16043 raft_consensus.cc:357] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:42.860153 16043 raft_consensus.cc:383] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.860273 16043 raft_consensus.cc:738] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.860632 16043 consensus_queue.cc:260] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:42.862030 16042 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:42.863700 16042 tablet_bootstrap.cc:654] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.863904 16043 ts_tablet_manager.cc:1428] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250905 07:50:42.868822 16043 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:42.871089 16043 tablet_bootstrap.cc:654] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.871696 16045 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:42.872489 16042 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:42.872705 16042 ts_tablet_manager.cc:1397] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20250905 07:50:42.873322 16043 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:42.873449 16043 ts_tablet_manager.cc:1397] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20250905 07:50:42.873574 16042 raft_consensus.cc:357] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.873787 16042 raft_consensus.cc:383] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.873867 16042 raft_consensus.cc:738] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.874121 16043 raft_consensus.cc:357] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.874089 16042 consensus_queue.cc:260] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.874257 16043 raft_consensus.cc:383] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.874351 16043 raft_consensus.cc:738] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.874574 16045 tablet_bootstrap.cc:654] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.874823 16042 ts_tablet_manager.cc:1428] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:50:42.874629 16043 consensus_queue.cc:260] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.875115 16042 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:42.875563 16043 ts_tablet_manager.cc:1428] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:50:42.878219 16042 tablet_bootstrap.cc:654] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.878461 16043 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:42.880219 16043 tablet_bootstrap.cc:654] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.881881 16043 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:42.882015 16043 ts_tablet_manager.cc:1397] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:50:42.882397 16045 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:42.882508 16045 ts_tablet_manager.cc:1397] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20250905 07:50:42.882560 16043 raft_consensus.cc:357] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.882704 16043 raft_consensus.cc:383] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.882762 16043 raft_consensus.cc:738] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.882952 16043 consensus_queue.cc:260] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.883057 16045 raft_consensus.cc:357] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:42.883252 16045 raft_consensus.cc:383] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.883319 16045 raft_consensus.cc:738] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.883415 16043 ts_tablet_manager.cc:1428] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:42.883553 16045 consensus_queue.cc:260] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:42.883697 16043 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:42.884292 16045 ts_tablet_manager.cc:1428] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:42.884593 16045 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:42.885468 16043 tablet_bootstrap.cc:654] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.886449 16045 tablet_bootstrap.cc:654] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.887147 16042 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:42.887147 16043 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:42.887293 16043 ts_tablet_manager.cc:1397] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:50:42.887295 16042 ts_tablet_manager.cc:1397] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.004s
I20250905 07:50:42.888422 16043 raft_consensus.cc:357] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.888558 16043 raft_consensus.cc:383] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.888608 16043 raft_consensus.cc:738] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.888780 16043 consensus_queue.cc:260] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.888823 16042 raft_consensus.cc:357] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.888927 16042 raft_consensus.cc:383] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.888978 16042 raft_consensus.cc:738] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.889138 16042 consensus_queue.cc:260] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.889312 16043 ts_tablet_manager.cc:1428] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:50:42.889540 16042 ts_tablet_manager.cc:1428] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250905 07:50:42.889760 16042 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:42.891557 16042 tablet_bootstrap.cc:654] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.891577 16045 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:42.891847 16045 ts_tablet_manager.cc:1397] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20250905 07:50:42.892598 16045 raft_consensus.cc:357] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.892818 16045 raft_consensus.cc:383] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.892930 16045 raft_consensus.cc:738] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.893146 16045 consensus_queue.cc:260] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.893512 16045 ts_tablet_manager.cc:1428] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:42.893812 16045 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:42.895195 16045 tablet_bootstrap.cc:654] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.896472 16042 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:42.896596 16042 ts_tablet_manager.cc:1397] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20250905 07:50:42.896924 16045 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:42.897053 16045 ts_tablet_manager.cc:1397] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20250905 07:50:42.897274 16042 raft_consensus.cc:357] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.897399 16042 raft_consensus.cc:383] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.897432 16042 raft_consensus.cc:738] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.897593 16042 consensus_queue.cc:260] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.897630 16045 raft_consensus.cc:357] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.897738 16045 raft_consensus.cc:383] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.897897 16045 raft_consensus.cc:738] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.897979 16042 ts_tablet_manager.cc:1428] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:42.898099 16045 consensus_queue.cc:260] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.898272 16042 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:42.899055 16045 ts_tablet_manager.cc:1428] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250905 07:50:42.899415 16045 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:42.900040 16042 tablet_bootstrap.cc:654] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.901528 16045 tablet_bootstrap.cc:654] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:42.902956 16042 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:42.903086 16042 ts_tablet_manager.cc:1397] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20250905 07:50:42.903669 16042 raft_consensus.cc:357] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:42.903826 16042 raft_consensus.cc:383] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.903918 16042 raft_consensus.cc:738] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.904127 16042 consensus_queue.cc:260] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:42.904453 16045 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:42.904527 16042 ts_tablet_manager.cc:1428] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250905 07:50:42.904567 16045 ts_tablet_manager.cc:1397] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20250905 07:50:42.905156 16045 raft_consensus.cc:357] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.905288 16045 raft_consensus.cc:383] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:42.905352 16045 raft_consensus.cc:738] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:42.905535 16045 consensus_queue.cc:260] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.905931 16045 ts_tablet_manager.cc:1428] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:42.916735 16053 raft_consensus.cc:491] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:42.916867 16053 raft_consensus.cc:513] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.917580 16053 leader_election.cc:290] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:42.925418 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:42.925418 15923 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:42.925704 15923 raft_consensus.cc:2466] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:42.925704 15997 raft_consensus.cc:2466] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:42.926158 15811 leader_election.cc:304] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:42.926402 16053 raft_consensus.cc:2802] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:42.926479 16053 raft_consensus.cc:491] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:42.926558 16053 raft_consensus.cc:3058] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:42.928462 16053 raft_consensus.cc:513] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.928963 16053 leader_election.cc:290] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Requested vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:42.929396 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:42.929397 15923 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
I20250905 07:50:42.929538 15997 raft_consensus.cc:3058] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:42.929566 15923 raft_consensus.cc:3058] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:42.931370 15923 raft_consensus.cc:2466] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:42.931380 15997 raft_consensus.cc:2466] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:42.931730 15811 leader_election.cc:304] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters:
I20250905 07:50:42.931985 16053 raft_consensus.cc:2802] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:42.932416 16053 raft_consensus.cc:695] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Becoming Leader. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Running, Role: LEADER
I20250905 07:50:42.932694 16053 consensus_queue.cc:237] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:42.935331 15746 catalog_manager.cc:5582] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad reported cstate change: term changed from 0 to 1, leader changed from <none> to 8673871edcb144dc863da6e6fb876fad (127.8.229.65). New cstate: current_term: 1 leader_uuid: "8673871edcb144dc863da6e6fb876fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:43.013600 16053 raft_consensus.cc:491] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.013793 16053 raft_consensus.cc:513] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.014485 16053 leader_election.cc:290] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.014948 15923 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:43.014971 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.015137 15923 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.015143 15997 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.015517 15811 leader_election.cc:304] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters:
I20250905 07:50:43.015733 16053 raft_consensus.cc:2802] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:43.015810 16053 raft_consensus.cc:491] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:43.015893 16053 raft_consensus.cc:3058] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.017705 16053 raft_consensus.cc:513] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.018196 16053 leader_election.cc:290] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Requested vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.018568 15923 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
I20250905 07:50:43.018695 15923 raft_consensus.cc:3058] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.018651 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:43.018819 15997 raft_consensus.cc:3058] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.020664 15997 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.020661 15923 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.021018 15812 leader_election.cc:304] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.021252 16053 raft_consensus.cc:2802] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:43.021369 16053 raft_consensus.cc:695] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Becoming Leader. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Running, Role: LEADER
I20250905 07:50:43.021615 16053 consensus_queue.cc:237] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.021799 16062 raft_consensus.cc:491] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.021953 16062 raft_consensus.cc:513] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.022670 16062 leader_election.cc:290] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.023263 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "135d618a63ea48448bf95be21e340ce5" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.023422 15997 raft_consensus.cc:2466] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.023726 15812 leader_election.cc:304] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.023841 15923 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "135d618a63ea48448bf95be21e340ce5" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:43.023957 16053 raft_consensus.cc:2802] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:43.024044 16053 raft_consensus.cc:491] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:43.024017 15923 raft_consensus.cc:2466] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.024127 16053 raft_consensus.cc:3058] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.024430 15746 catalog_manager.cc:5582] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad reported cstate change: term changed from 0 to 1, leader changed from <none> to 8673871edcb144dc863da6e6fb876fad (127.8.229.65). New cstate: current_term: 1 leader_uuid: "8673871edcb144dc863da6e6fb876fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:43.035200 16053 raft_consensus.cc:513] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.035820 16053 leader_election.cc:290] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Requested vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.036329 15923 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "135d618a63ea48448bf95be21e340ce5" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
I20250905 07:50:43.036330 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "135d618a63ea48448bf95be21e340ce5" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:43.036495 15923 raft_consensus.cc:3058] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.036518 15997 raft_consensus.cc:3058] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.038363 15923 raft_consensus.cc:2466] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.038360 15997 raft_consensus.cc:2466] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.038779 15812 leader_election.cc:304] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.039067 16053 raft_consensus.cc:2802] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:43.039183 16053 raft_consensus.cc:695] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Becoming Leader. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Running, Role: LEADER
I20250905 07:50:43.039407 16053 consensus_queue.cc:237] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.041980 15746 catalog_manager.cc:5582] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad reported cstate change: term changed from 0 to 1, leader changed from <none> to 8673871edcb144dc863da6e6fb876fad (127.8.229.65). New cstate: current_term: 1 leader_uuid: "8673871edcb144dc863da6e6fb876fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:43.297122 16046 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.297326 16046 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:43.297331 16053 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.297537 16053 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:43.297930 16049 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.298085 16049 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:43.298202 16046 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795), 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917)
I20250905 07:50:43.298365 16053 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:43.298662 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.298765 16049 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:43.298870 15997 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.299314 15812 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.300634 16053 raft_consensus.cc:2802] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:43.300721 16053 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:43.300792 16053 raft_consensus.cc:3058] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.303592 16053 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:43.304173 16053 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Requested vote from peers a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:43.304517 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:43.304692 15997 raft_consensus.cc:3058] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.304719 15923 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:43.304904 15923 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.304919 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
I20250905 07:50:43.305086 15922 raft_consensus.cc:3058] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.307693 15997 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.308020 15812 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.309424 15922 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.310048 16053 raft_consensus.cc:2802] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:43.310247 16053 raft_consensus.cc:695] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Becoming Leader. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Running, Role: LEADER
I20250905 07:50:43.310561 16053 consensus_queue.cc:237] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:43.311220 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:43.312474 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:43.314031 15746 catalog_manager.cc:5582] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad reported cstate change: term changed from 0 to 1, leader changed from <none> to 8673871edcb144dc863da6e6fb876fad (127.8.229.65). New cstate: current_term: 1 leader_uuid: "8673871edcb144dc863da6e6fb876fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:43.315577 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:43.315760 15922 raft_consensus.cc:2391] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6f256ebcfff460ea87eaa92563a9231 in current term 1: Already voted for candidate 8673871edcb144dc863da6e6fb876fad in this term.
I20250905 07:50:43.316262 15960 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6f256ebcfff460ea87eaa92563a9231; no voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad
I20250905 07:50:43.316531 16049 raft_consensus.cc:2747] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:50:43.318123 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.318271 15997 raft_consensus.cc:2391] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 448a1f2f153145c0aa61c7cf3ed83d27 in current term 1: Already voted for candidate 8673871edcb144dc863da6e6fb876fad in this term.
I20250905 07:50:43.318559 15887 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27; no voters: 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231
I20250905 07:50:43.318773 16046 raft_consensus.cc:2747] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250905 07:50:43.371752 9109 test_util.cc:276] Using random seed: 407480022
I20250905 07:50:43.374289 16053 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.378971 16064 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.387259 15746 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40006:
name: "dugtrio"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250905 07:50:43.387912 15746 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table dugtrio in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250905 07:50:43.395557 15838 tablet_service.cc:1468] Processing CreateTablet for tablet a0fba91ca8d143f39bd2d3cf76cfc13f (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:43.395896 15910 tablet_service.cc:1468] Processing CreateTablet for tablet a0fba91ca8d143f39bd2d3cf76cfc13f (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:43.396025 15838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0fba91ca8d143f39bd2d3cf76cfc13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.395946 15836 tablet_service.cc:1468] Processing CreateTablet for tablet 0df306fbdd3a4d1aa71e2bc678d35465 (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:43.396250 15910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0fba91ca8d143f39bd2d3cf76cfc13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.396476 15836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df306fbdd3a4d1aa71e2bc678d35465. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.399961 15912 tablet_service.cc:1468] Processing CreateTablet for tablet 0df306fbdd3a4d1aa71e2bc678d35465 (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:43.400315 15912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df306fbdd3a4d1aa71e2bc678d35465. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.401209 16045 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:43.402376 16045 tablet_bootstrap.cc:654] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.403971 16045 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:43.404074 15985 tablet_service.cc:1468] Processing CreateTablet for tablet a0fba91ca8d143f39bd2d3cf76cfc13f (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250905 07:50:43.404438 15985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0fba91ca8d143f39bd2d3cf76cfc13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.404094 16045 ts_tablet_manager.cc:1397] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20250905 07:50:43.405287 15838 tablet_service.cc:1468] Processing CreateTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:43.405432 16045 raft_consensus.cc:357] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.405591 16045 raft_consensus.cc:383] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.405654 16045 raft_consensus.cc:738] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.405599 15838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ab5d983123841ee8cdf655cc05fe09d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.406221 15987 tablet_service.cc:1468] Processing CreateTablet for tablet 0df306fbdd3a4d1aa71e2bc678d35465 (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250905 07:50:43.406554 15987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df306fbdd3a4d1aa71e2bc678d35465. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.411559 16045 consensus_queue.cc:260] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.413457 15913 tablet_service.cc:1468] Processing CreateTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:43.415115 15913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ab5d983123841ee8cdf655cc05fe09d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.415567 16045 ts_tablet_manager.cc:1428] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20250905 07:50:43.415377 16085 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:43.415969 16045 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:43.417276 16045 tablet_bootstrap.cc:654] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.417481 16085 tablet_bootstrap.cc:654] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.419245 16086 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:43.421794 16086 tablet_bootstrap.cc:654] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.425344 15985 tablet_service.cc:1468] Processing CreateTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250905 07:50:43.425731 15985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ab5d983123841ee8cdf655cc05fe09d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.425787 16085 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:43.425908 16085 ts_tablet_manager.cc:1397] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20250905 07:50:43.426669 16085 raft_consensus.cc:357] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.426836 16085 raft_consensus.cc:383] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.426893 16085 raft_consensus.cc:738] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.426944 15912 tablet_service.cc:1468] Processing CreateTablet for tablet 257f5535e785474f9d4f2eaf93040ba0 (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:43.427090 16085 consensus_queue.cc:260] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.427368 15912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257f5535e785474f9d4f2eaf93040ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.425911 15838 tablet_service.cc:1468] Processing CreateTablet for tablet 257f5535e785474f9d4f2eaf93040ba0 (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:43.427647 15838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257f5535e785474f9d4f2eaf93040ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.434696 16085 ts_tablet_manager.cc:1428] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20250905 07:50:43.436627 15985 tablet_service.cc:1468] Processing CreateTablet for tablet 257f5535e785474f9d4f2eaf93040ba0 (DEFAULT_TABLE table=dugtrio [id=1504cfbc6e914551b346d3724cb098b1]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250905 07:50:43.436834 16086 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:43.436968 16086 ts_tablet_manager.cc:1397] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.006s
I20250905 07:50:43.437166 15985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257f5535e785474f9d4f2eaf93040ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:43.437703 16086 raft_consensus.cc:357] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.437872 16086 raft_consensus.cc:383] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.437943 16086 raft_consensus.cc:738] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.438198 16086 consensus_queue.cc:260] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.440163 16085 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:43.440658 16064 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.440814 16086 ts_tablet_manager.cc:1428] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20250905 07:50:43.441699 16085 tablet_bootstrap.cc:654] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.443084 16086 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:43.443400 16085 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:43.443538 16085 ts_tablet_manager.cc:1397] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20250905 07:50:43.444207 16085 raft_consensus.cc:357] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.444372 16085 raft_consensus.cc:383] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.444432 16085 raft_consensus.cc:738] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.444638 16085 consensus_queue.cc:260] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.445086 16085 ts_tablet_manager.cc:1428] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250905 07:50:43.445322 16085 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:43.445402 16086 tablet_bootstrap.cc:654] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.446663 16085 tablet_bootstrap.cc:654] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.448925 16085 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:43.449054 16085 ts_tablet_manager.cc:1397] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20250905 07:50:43.449779 16085 raft_consensus.cc:357] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.449926 16085 raft_consensus.cc:383] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.450004 16086 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:43.449992 16085 raft_consensus.cc:738] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.450129 16086 ts_tablet_manager.cc:1397] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250905 07:50:43.450230 16064 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.450816 16086 raft_consensus.cc:357] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.450965 16086 raft_consensus.cc:383] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.451020 16086 raft_consensus.cc:738] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.451576 16086 consensus_queue.cc:260] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.452293 16085 consensus_queue.cc:260] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.452879 16085 ts_tablet_manager.cc:1428] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20250905 07:50:43.452972 16045 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:43.453156 16085 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: Bootstrap starting.
I20250905 07:50:43.453100 16045 ts_tablet_manager.cc:1397] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.037s user 0.006s sys 0.011s
I20250905 07:50:43.453938 16045 raft_consensus.cc:357] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.454057 16045 raft_consensus.cc:383] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.454116 16045 raft_consensus.cc:738] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.454308 16045 consensus_queue.cc:260] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.454653 16085 tablet_bootstrap.cc:654] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.454811 16045 ts_tablet_manager.cc:1428] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250905 07:50:43.455065 16045 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:43.456984 16086 ts_tablet_manager.cc:1428] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20250905 07:50:43.457300 16086 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:43.458832 16086 tablet_bootstrap.cc:654] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.460330 16045 tablet_bootstrap.cc:654] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.461673 16045 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:43.461792 16045 ts_tablet_manager.cc:1397] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250905 07:50:43.462522 16045 raft_consensus.cc:357] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.462664 16045 raft_consensus.cc:383] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.462723 16045 raft_consensus.cc:738] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.462941 16045 consensus_queue.cc:260] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.463711 16085 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: No bootstrap required, opened a new log
I20250905 07:50:43.463873 16085 ts_tablet_manager.cc:1397] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20250905 07:50:43.464056 16086 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:43.464143 16045 ts_tablet_manager.cc:1428] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250905 07:50:43.464176 16086 ts_tablet_manager.cc:1397] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20250905 07:50:43.464424 16045 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:43.464797 16085 raft_consensus.cc:357] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.464855 16086 raft_consensus.cc:357] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.464996 16085 raft_consensus.cc:383] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.465183 16085 raft_consensus.cc:738] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.465034 16086 raft_consensus.cc:383] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.465317 16086 raft_consensus.cc:738] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.465517 16085 consensus_queue.cc:260] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.465549 16086 consensus_queue.cc:260] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.466059 16085 ts_tablet_manager.cc:1428] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250905 07:50:43.466141 16086 ts_tablet_manager.cc:1428] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250905 07:50:43.466403 16086 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: Bootstrap starting.
I20250905 07:50:43.467133 16045 tablet_bootstrap.cc:654] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.467744 16086 tablet_bootstrap.cc:654] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: Neither blocks nor log segments found. Creating new log.
I20250905 07:50:43.472211 16086 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: No bootstrap required, opened a new log
I20250905 07:50:43.472237 16045 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: No bootstrap required, opened a new log
I20250905 07:50:43.472373 16045 ts_tablet_manager.cc:1397] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20250905 07:50:43.472396 16086 ts_tablet_manager.cc:1397] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20250905 07:50:43.472956 16086 raft_consensus.cc:357] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.473084 16086 raft_consensus.cc:383] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.473006 16045 raft_consensus.cc:357] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.473141 16086 raft_consensus.cc:738] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.473234 16045 raft_consensus.cc:383] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250905 07:50:43.473320 16045 raft_consensus.cc:738] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:43.473367 16086 consensus_queue.cc:260] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.473527 16045 consensus_queue.cc:260] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.473847 16086 ts_tablet_manager.cc:1428] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:43.473980 16045 ts_tablet_manager.cc:1428] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:50:43.487972 16046 raft_consensus.cc:491] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.488149 16046 raft_consensus.cc:513] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.488726 16046 leader_election.cc:290] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.489096 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4ab5d983123841ee8cdf655cc05fe09d" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:43.489145 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4ab5d983123841ee8cdf655cc05fe09d" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.489313 15848 raft_consensus.cc:2466] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 0.
I20250905 07:50:43.489324 15997 raft_consensus.cc:2466] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 0.
I20250905 07:50:43.489687 15889 leader_election.cc:304] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.489939 16046 raft_consensus.cc:2802] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:43.490036 16046 raft_consensus.cc:491] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:43.490132 16046 raft_consensus.cc:3058] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.492076 16046 raft_consensus.cc:513] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.492599 16046 leader_election.cc:290] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 election: Requested vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.492964 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4ab5d983123841ee8cdf655cc05fe09d" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad"
I20250905 07:50:43.492964 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4ab5d983123841ee8cdf655cc05fe09d" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:43.493167 15997 raft_consensus.cc:3058] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.493184 15848 raft_consensus.cc:3058] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.495939 15848 raft_consensus.cc:2466] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:43.495939 15997 raft_consensus.cc:2466] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:43.496316 15887 leader_election.cc:304] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.496562 16046 raft_consensus.cc:2802] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:43.496953 16046 raft_consensus.cc:695] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 LEADER]: Becoming Leader. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Running, Role: LEADER
I20250905 07:50:43.497200 16046 consensus_queue.cc:237] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.499486 15746 catalog_manager.cc:5582] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
W20250905 07:50:43.522817 15794 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250905 07:50:43.529383 16046 raft_consensus.cc:491] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.529598 16046 raft_consensus.cc:513] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.530206 16046 leader_election.cc:290] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.530732 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0df306fbdd3a4d1aa71e2bc678d35465" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.530740 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0df306fbdd3a4d1aa71e2bc678d35465" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:43.530927 15997 raft_consensus.cc:2466] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 0.
I20250905 07:50:43.531026 15848 raft_consensus.cc:2466] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 0.
I20250905 07:50:43.531338 15887 leader_election.cc:304] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.531595 16046 raft_consensus.cc:2802] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:43.531697 16046 raft_consensus.cc:491] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:43.531787 16046 raft_consensus.cc:3058] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.533850 16046 raft_consensus.cc:513] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.534327 16046 leader_election.cc:290] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 election: Requested vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.534708 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0df306fbdd3a4d1aa71e2bc678d35465" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad"
I20250905 07:50:43.534823 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0df306fbdd3a4d1aa71e2bc678d35465" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:43.534888 15848 raft_consensus.cc:3058] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.535002 15997 raft_consensus.cc:3058] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.537679 15848 raft_consensus.cc:2466] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:43.537679 15997 raft_consensus.cc:2466] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:43.538094 15889 leader_election.cc:304] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters:
I20250905 07:50:43.538338 16046 raft_consensus.cc:2802] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:43.538272 16064 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.538496 16046 raft_consensus.cc:695] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 LEADER]: Becoming Leader. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Running, Role: LEADER
I20250905 07:50:43.538748 16046 consensus_queue.cc:237] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.542352 16104 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.542173 15743 catalog_manager.cc:5582] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:43.611691 16062 raft_consensus.cc:491] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.611923 16062 raft_consensus.cc:513] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.612684 16062 leader_election.cc:290] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.613109 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "257f5535e785474f9d4f2eaf93040ba0" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:43.613181 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "257f5535e785474f9d4f2eaf93040ba0" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.613303 15922 raft_consensus.cc:2466] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.613416 15997 raft_consensus.cc:2466] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 0.
I20250905 07:50:43.613670 15811 leader_election.cc:304] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters:
I20250905 07:50:43.613910 16062 raft_consensus.cc:2802] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:43.614012 16062 raft_consensus.cc:491] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:43.614111 16062 raft_consensus.cc:3058] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.616079 16062 raft_consensus.cc:513] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.616685 16062 leader_election.cc:290] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Requested vote from peers 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.617079 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "257f5535e785474f9d4f2eaf93040ba0" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
I20250905 07:50:43.617254 15922 raft_consensus.cc:3058] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.617249 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "257f5535e785474f9d4f2eaf93040ba0" candidate_uuid: "8673871edcb144dc863da6e6fb876fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:43.617432 15997 raft_consensus.cc:3058] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.619235 15922 raft_consensus.cc:2466] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.619406 15997 raft_consensus.cc:2466] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8673871edcb144dc863da6e6fb876fad in term 1.
I20250905 07:50:43.619608 15811 leader_election.cc:304] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters:
I20250905 07:50:43.619879 16062 raft_consensus.cc:2802] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:43.620030 16062 raft_consensus.cc:695] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Becoming Leader. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Running, Role: LEADER
I20250905 07:50:43.620303 16062 consensus_queue.cc:237] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.622843 15743 catalog_manager.cc:5582] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad reported cstate change: term changed from 0 to 1, leader changed from <none> to 8673871edcb144dc863da6e6fb876fad (127.8.229.65). New cstate: current_term: 1 leader_uuid: "8673871edcb144dc863da6e6fb876fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:43.809007 16062 consensus_queue.cc:1035] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.812973 16053 consensus_queue.cc:1035] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.871210 16046 raft_consensus.cc:491] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:43.871474 16046 raft_consensus.cc:513] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.872099 16046 leader_election.cc:290] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.872534 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a0fba91ca8d143f39bd2d3cf76cfc13f" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:43.872534 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a0fba91ca8d143f39bd2d3cf76cfc13f" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:43.872776 15997 raft_consensus.cc:2466] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 0.
I20250905 07:50:43.872823 15848 raft_consensus.cc:2466] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 0.
I20250905 07:50:43.873198 15887 leader_election.cc:304] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:43.873505 16046 raft_consensus.cc:2802] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250905 07:50:43.873610 16046 raft_consensus.cc:491] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250905 07:50:43.873700 16046 raft_consensus.cc:3058] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.875705 16046 raft_consensus.cc:513] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.876204 16046 leader_election.cc:290] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 election: Requested vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:43.876605 15848 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a0fba91ca8d143f39bd2d3cf76cfc13f" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad"
I20250905 07:50:43.876614 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a0fba91ca8d143f39bd2d3cf76cfc13f" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:43.876753 15848 raft_consensus.cc:3058] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.876796 15997 raft_consensus.cc:3058] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 0 FOLLOWER]: Advancing to term 1
I20250905 07:50:43.878643 15848 raft_consensus.cc:2466] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:43.878638 15997 raft_consensus.cc:2466] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:43.879001 15889 leader_election.cc:304] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters:
I20250905 07:50:43.879298 16046 raft_consensus.cc:2802] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader election won for term 1
I20250905 07:50:43.879477 16046 raft_consensus.cc:695] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 LEADER]: Becoming Leader. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Running, Role: LEADER
I20250905 07:50:43.879724 16046 consensus_queue.cc:237] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:43.882112 15743 catalog_manager.cc:5582] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66). New cstate: current_term: 1 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:43.929052 16046 consensus_queue.cc:1035] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.933300 16046 consensus_queue.cc:1035] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:43.970113 9109 tablet_server.cc:178] TabletServer@127.8.229.65:0 shutting down...
I20250905 07:50:43.979924 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:43.980257 9109 tablet_replica.cc:331] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.980463 9109 raft_consensus.cc:2241] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:43.980710 9109 pending_rounds.cc:70] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Trying to abort 1 pending ops.
I20250905 07:50:43.980773 9109 pending_rounds.cc:77] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912204271247360 op_type: NO_OP noop_request { }
I20250905 07:50:43.980913 9109 raft_consensus.cc:2887] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:43.981045 9109 raft_consensus.cc:2270] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.981815 9109 tablet_replica.cc:331] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.981978 9109 raft_consensus.cc:2241] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:43.982153 9109 raft_consensus.cc:2270] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.982765 9109 tablet_replica.cc:331] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.982918 9109 raft_consensus.cc:2241] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:43.983206 9109 raft_consensus.cc:2270] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.983795 9109 tablet_replica.cc:331] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.983959 9109 raft_consensus.cc:2241] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:43.984097 9109 raft_consensus.cc:2270] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.984513 9109 tablet_replica.cc:331] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.984658 9109 raft_consensus.cc:2241] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:43.984944 9109 raft_consensus.cc:2270] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.985482 9109 tablet_replica.cc:331] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.985654 9109 raft_consensus.cc:2241] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:43.985901 9109 raft_consensus.cc:2270] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.986521 9109 tablet_replica.cc:331] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.986662 9109 raft_consensus.cc:2241] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:43.986917 9109 raft_consensus.cc:2270] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.987639 9109 tablet_replica.cc:331] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: stopping tablet replica
I20250905 07:50:43.987810 9109 raft_consensus.cc:2241] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:43.987965 9109 raft_consensus.cc:2270] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:43.993592 9109 tablet_server.cc:195] TabletServer@127.8.229.65:0 shutdown complete.
W20250905 07:50:44.006875 15889 consensus_peers.cc:489] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 -> Peer 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): Couldn't send request to peer 8673871edcb144dc863da6e6fb876fad. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:44.066799 16046 consensus_queue.cc:1035] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250905 07:50:44.284050 16111 consensus_queue.cc:1035] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250905 07:50:44.289136 15889 consensus_peers.cc:489] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 -> Peer 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): Couldn't send request to peer 8673871edcb144dc863da6e6fb876fad. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250905 07:50:44.361907 15889 consensus_peers.cc:489] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 -> Peer 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): Couldn't send request to peer 8673871edcb144dc863da6e6fb876fad. Status: Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:44.948864 16100 raft_consensus.cc:491] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:44.949066 16100 raft_consensus.cc:513] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:44.949708 16100 leader_election.cc:290] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:44.950258 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
W20250905 07:50:44.950954 15889 leader_election.cc:336] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111)
I20250905 07:50:44.951045 15889 leader_election.cc:304] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27; no voters: 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231
I20250905 07:50:44.951221 16100 raft_consensus.cc:2747] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250905 07:50:44.973214 16100 consensus_queue.cc:579] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Leader has been unable to successfully communicate with peer 8673871edcb144dc863da6e6fb876fad for more than 1 seconds (1.040s)
I20250905 07:50:44.981292 16100 consensus_queue.cc:579] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Leader has been unable to successfully communicate with peer 8673871edcb144dc863da6e6fb876fad for more than 1 seconds (1.442s)
I20250905 07:50:45.012869 15744 catalog_manager.cc:2482] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40004:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20250905 07:50:45.013051 15744 catalog_manager.cc:2730] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40004:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20250905 07:50:45.017892 15744 catalog_manager.cc:5869] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: Sending DeleteTablet for 3 replicas of tablet a0fba91ca8d143f39bd2d3cf76cfc13f
I20250905 07:50:45.018457 15744 catalog_manager.cc:5869] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: Sending DeleteTablet for 3 replicas of tablet 0df306fbdd3a4d1aa71e2bc678d35465
I20250905 07:50:45.018821 15913 tablet_service.cc:1515] Processing DeleteTablet for tablet a0fba91ca8d143f39bd2d3cf76cfc13f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:41204
I20250905 07:50:45.018909 15744 catalog_manager.cc:5869] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: Sending DeleteTablet for 3 replicas of tablet 4ab5d983123841ee8cdf655cc05fe09d
I20250905 07:50:45.019218 15985 tablet_service.cc:1515] Processing DeleteTablet for tablet a0fba91ca8d143f39bd2d3cf76cfc13f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:52330
I20250905 07:50:45.019333 15744 catalog_manager.cc:5869] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: Sending DeleteTablet for 3 replicas of tablet 257f5535e785474f9d4f2eaf93040ba0
I20250905 07:50:45.019722 15987 tablet_service.cc:1515] Processing DeleteTablet for tablet 0df306fbdd3a4d1aa71e2bc678d35465 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:52330
I20250905 07:50:45.019796 15912 tablet_service.cc:1515] Processing DeleteTablet for tablet 0df306fbdd3a4d1aa71e2bc678d35465 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:41204
I20250905 07:50:45.019954 15911 tablet_service.cc:1515] Processing DeleteTablet for tablet 257f5535e785474f9d4f2eaf93040ba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:41204
I20250905 07:50:45.020000 15910 tablet_service.cc:1515] Processing DeleteTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:41204
I20250905 07:50:45.020440 15986 tablet_service.cc:1515] Processing DeleteTablet for tablet 257f5535e785474f9d4f2eaf93040ba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:52330
I20250905 07:50:45.020440 15987 tablet_service.cc:1515] Processing DeleteTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:52330
I20250905 07:50:45.021232 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250905 07:50:45.029618 16128 tablet_replica.cc:331] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:45.029811 16127 tablet_replica.cc:331] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:45.030052 16127 raft_consensus.cc:2241] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:45.030486 16127 raft_consensus.cc:2270] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250905 07:50:45.031652 16130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:45.031780 16127 ts_tablet_manager.cc:1905] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250905 07:50:45.032516 16131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:45.029834 16128 raft_consensus.cc:2241] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.033208 16128 raft_consensus.cc:2270] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250905 07:50:45.033867 15733 catalog_manager.cc:4671] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0df306fbdd3a4d1aa71e2bc678d35465: Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111)
I20250905 07:50:45.034317 16128 ts_tablet_manager.cc:1905] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.040102 16127 ts_tablet_manager.cc:1918] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.040343 16127 log.cc:1199] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/a0fba91ca8d143f39bd2d3cf76cfc13f
I20250905 07:50:45.040916 16127 ts_tablet_manager.cc:1939] T a0fba91ca8d143f39bd2d3cf76cfc13f P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting consensus metadata
I20250905 07:50:45.042618 15730 catalog_manager.cc:4928] TS 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761): tablet a0fba91ca8d143f39bd2d3cf76cfc13f (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.043038 16127 tablet_replica.cc:331] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:45.043164 16128 ts_tablet_manager.cc:1918] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.043359 16128 log.cc:1199] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/a0fba91ca8d143f39bd2d3cf76cfc13f
I20250905 07:50:45.043395 16127 raft_consensus.cc:2241] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:45.043862 16127 raft_consensus.cc:2270] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.044025 16128 ts_tablet_manager.cc:1939] T a0fba91ca8d143f39bd2d3cf76cfc13f P a6f256ebcfff460ea87eaa92563a9231: Deleting consensus metadata
I20250905 07:50:45.043987 16100 raft_consensus.cc:491] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.044201 16100 raft_consensus.cc:513] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.044838 16100 leader_election.cc:290] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:45.045094 16127 ts_tablet_manager.cc:1905] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.045187 15731 catalog_manager.cc:4928] TS a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795): tablet a0fba91ca8d143f39bd2d3cf76cfc13f (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.045531 16128 tablet_replica.cc:331] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:45.045771 16128 raft_consensus.cc:2241] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.045964 16128 raft_consensus.cc:2270] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.045966 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "135d618a63ea48448bf95be21e340ce5" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:45.046195 15997 raft_consensus.cc:2466] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:45.046591 15887 leader_election.cc:304] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:45.047149 16128 ts_tablet_manager.cc:1905] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250905 07:50:45.047710 15889 leader_election.cc:336] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111)
I20250905 07:50:45.048022 16100 raft_consensus.cc:2802] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:45.048138 16100 raft_consensus.cc:491] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.048218 16100 raft_consensus.cc:3058] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.050767 16100 raft_consensus.cc:513] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
W20250905 07:50:45.052642 15889 leader_election.cc:336] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111)
I20250905 07:50:45.052891 16127 ts_tablet_manager.cc:1918] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.053015 16127 log.cc:1199] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/0df306fbdd3a4d1aa71e2bc678d35465
I20250905 07:50:45.053365 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "135d618a63ea48448bf95be21e340ce5" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:45.053463 16127 ts_tablet_manager.cc:1939] T 0df306fbdd3a4d1aa71e2bc678d35465 P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting consensus metadata
I20250905 07:50:45.053544 15997 raft_consensus.cc:3058] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.053664 16128 ts_tablet_manager.cc:1918] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.053797 16128 log.cc:1199] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/0df306fbdd3a4d1aa71e2bc678d35465
I20250905 07:50:45.054200 16128 ts_tablet_manager.cc:1939] T 0df306fbdd3a4d1aa71e2bc678d35465 P a6f256ebcfff460ea87eaa92563a9231: Deleting consensus metadata
I20250905 07:50:45.054594 15730 catalog_manager.cc:4928] TS 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761): tablet 0df306fbdd3a4d1aa71e2bc678d35465 (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.055130 15731 catalog_manager.cc:4928] TS a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795): tablet 0df306fbdd3a4d1aa71e2bc678d35465 (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.055343 16127 tablet_replica.cc:331] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:45.055545 16128 tablet_replica.cc:331] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:45.055591 16127 raft_consensus.cc:2241] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.055760 16127 raft_consensus.cc:2270] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.055779 16128 raft_consensus.cc:2241] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.055977 16128 raft_consensus.cc:2270] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.056475 16127 ts_tablet_manager.cc:1905] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.056209 15997 raft_consensus.cc:2466] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 2.
I20250905 07:50:45.056614 16128 ts_tablet_manager.cc:1905] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.056720 16135 raft_consensus.cc:491] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250905 07:50:45.056866 16135 raft_consensus.cc:513] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.056957 15887 leader_election.cc:304] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, a6f256ebcfff460ea87eaa92563a9231; no voters: 8673871edcb144dc863da6e6fb876fad
I20250905 07:50:45.057212 16118 raft_consensus.cc:2802] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:45.057241 16100 leader_election.cc:290] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 election: Requested vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:45.057363 16118 raft_consensus.cc:695] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Becoming Leader. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Running, Role: LEADER
I20250905 07:50:45.057662 16118 consensus_queue.cc:237] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.061085 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "135d618a63ea48448bf95be21e340ce5" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
W20250905 07:50:45.061478 16133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:45.061894 16135 leader_election.cc:290] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:45.063849 16128 ts_tablet_manager.cc:1918] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250905 07:50:45.063989 16128 log.cc:1199] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/257f5535e785474f9d4f2eaf93040ba0
I20250905 07:50:45.064368 16128 ts_tablet_manager.cc:1939] T 257f5535e785474f9d4f2eaf93040ba0 P a6f256ebcfff460ea87eaa92563a9231: Deleting consensus metadata
I20250905 07:50:45.065205 16128 tablet_replica.cc:331] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:45.065412 16127 ts_tablet_manager.cc:1918] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250905 07:50:45.065510 16127 log.cc:1199] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/257f5535e785474f9d4f2eaf93040ba0
I20250905 07:50:45.065795 16127 ts_tablet_manager.cc:1939] T 257f5535e785474f9d4f2eaf93040ba0 P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting consensus metadata
I20250905 07:50:45.066489 16127 tablet_replica.cc:331] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:45.066533 15730 catalog_manager.cc:4928] TS 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761): tablet 257f5535e785474f9d4f2eaf93040ba0 (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.066649 16127 raft_consensus.cc:2241] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:45.066900 16127 raft_consensus.cc:2270] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.067418 16127 ts_tablet_manager.cc:1905] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.067673 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:45.068073 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:45.068162 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:45.068209 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058645068205 us; error 0 us; skew 500 ppm
I20250905 07:50:45.068434 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:45.070067 9109 webserver.cc:480] Webserver started at http://127.8.229.68:37763/ using document root <none> and password file <none>
I20250905 07:50:45.070389 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:45.070474 9109 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250905 07:50:45.070636 9109 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250905 07:50:45.065431 16128 raft_consensus.cc:2241] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.071532 9109 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-3-root/instance:
uuid: "742142751ce0432db4d94405382c6dc6"
format_stamp: "Formatted at 2025-09-05 07:50:45 on dist-test-slave-pgzt"
I20250905 07:50:45.071619 16128 raft_consensus.cc:2270] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.072315 15731 catalog_manager.cc:4928] TS a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795): tablet 257f5535e785474f9d4f2eaf93040ba0 (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.072335 16128 ts_tablet_manager.cc:1905] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.072737 16127 ts_tablet_manager.cc:1918] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.072875 16127 log.cc:1199] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/4ab5d983123841ee8cdf655cc05fe09d
I20250905 07:50:45.073277 16127 ts_tablet_manager.cc:1939] T 4ab5d983123841ee8cdf655cc05fe09d P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting consensus metadata
I20250905 07:50:45.074211 15730 catalog_manager.cc:4928] TS 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761): tablet 4ab5d983123841ee8cdf655cc05fe09d (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.077313 16128 ts_tablet_manager.cc:1918] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.077427 16128 log.cc:1199] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/4ab5d983123841ee8cdf655cc05fe09d
I20250905 07:50:45.077713 16128 ts_tablet_manager.cc:1939] T 4ab5d983123841ee8cdf655cc05fe09d P a6f256ebcfff460ea87eaa92563a9231: Deleting consensus metadata
I20250905 07:50:45.079030 9109 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.000s sys 0.005s
I20250905 07:50:45.079830 15731 catalog_manager.cc:4928] TS a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795): tablet 4ab5d983123841ee8cdf655cc05fe09d (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
W20250905 07:50:45.080863 15963 leader_election.cc:336] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): Network error: Client connection negotiation failed: client connection to 127.8.229.65:44917: connect: Connection refused (error 111)
I20250905 07:50:45.081002 15963 leader_election.cc:304] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6f256ebcfff460ea87eaa92563a9231; no voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad
I20250905 07:50:45.081243 16135 raft_consensus.cc:2747] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250905 07:50:45.081130 15744 catalog_manager.cc:5582] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: term changed from 1 to 2, leader changed from 8673871edcb144dc863da6e6fb876fad (127.8.229.65) to 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66). New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:45.081676 16142 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:45.082046 9109 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:45.082216 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-3-root
uuid: "742142751ce0432db4d94405382c6dc6"
format_stamp: "Formatted at 2025-09-05 07:50:45 on dist-test-slave-pgzt"
I20250905 07:50:45.082396 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:45.098203 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:45.098660 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:45.099252 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:45.100119 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250905 07:50:45.100172 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:45.100236 9109 ts_tablet_manager.cc:610] Registered 0 tablets
I20250905 07:50:45.100283 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:45.111546 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.68:33779
I20250905 07:50:45.111577 16204 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.68:33779 every 8 connection(s)
I20250905 07:50:45.112931 9109 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250905 07:50:45.115514 16208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:45.115629 16209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250905 07:50:45.116549 16211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250905 07:50:45.116588 9109 server_base.cc:1047] running on GCE node
I20250905 07:50:45.116997 9109 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250905 07:50:45.117060 9109 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250905 07:50:45.117115 9109 hybrid_clock.cc:648] HybridClock initialized: now 1757058645117111 us; error 0 us; skew 500 ppm
I20250905 07:50:45.117272 9109 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250905 07:50:45.118582 9109 webserver.cc:480] Webserver started at http://127.8.229.65:40795/ using document root <none> and password file <none>
I20250905 07:50:45.118886 9109 fs_manager.cc:362] Metadata directory not provided
I20250905 07:50:45.118945 9109 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250905 07:50:45.119379 16205 heartbeater.cc:344] Connected to a master server at 127.8.229.126:34109
I20250905 07:50:45.119524 16205 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:45.119903 16205 heartbeater.cc:507] Master 127.8.229.126:34109 requested a full tablet report, sending...
I20250905 07:50:45.120746 15744 ts_manager.cc:194] Registered new tserver with Master: 742142751ce0432db4d94405382c6dc6 (127.8.229.68:33779)
I20250905 07:50:45.121270 9109 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20250905 07:50:45.121650 15744 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40012
I20250905 07:50:45.123268 16216 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250905 07:50:45.123639 9109 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20250905 07:50:45.123737 9109 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root
uuid: "8673871edcb144dc863da6e6fb876fad"
format_stamp: "Formatted at 2025-09-05 07:50:41 on dist-test-slave-pgzt"
I20250905 07:50:45.123847 9109 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250905 07:50:45.134153 9109 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250905 07:50:45.134627 9109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250905 07:50:45.135130 9109 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250905 07:50:45.136399 16223 ts_tablet_manager.cc:542] Loading tablet metadata (0/8 complete)
I20250905 07:50:45.151257 9109 ts_tablet_manager.cc:579] Loaded tablet metadata (8 total tablets, 8 live tablets)
I20250905 07:50:45.151342 9109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20250905 07:50:45.151409 9109 ts_tablet_manager.cc:594] Registering tablets (0/8 complete)
I20250905 07:50:45.153184 16223 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.157594 16223 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250905 07:50:45.158025 16223 tablet_bootstrap.cc:492] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.158252 16223 ts_tablet_manager.cc:1397] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250905 07:50:45.158929 16223 raft_consensus.cc:357] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.159173 16223 raft_consensus.cc:738] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.159399 16223 consensus_queue.cc:260] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.160111 16223 ts_tablet_manager.cc:1428] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:45.160328 16223 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.164778 16223 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:45.165179 16223 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.165426 16223 ts_tablet_manager.cc:1397] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250905 07:50:45.166067 16223 raft_consensus.cc:357] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.166244 16223 raft_consensus.cc:738] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.166460 16223 consensus_queue.cc:260] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.166857 16223 ts_tablet_manager.cc:1428] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:45.167100 16223 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.169468 9109 ts_tablet_manager.cc:610] Registered 8 tablets
I20250905 07:50:45.169543 9109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.018s user 0.010s sys 0.006s
I20250905 07:50:45.170723 16223 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:45.171077 16223 tablet_bootstrap.cc:492] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.171301 16223 ts_tablet_manager.cc:1397] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250905 07:50:45.172040 16223 raft_consensus.cc:357] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.172242 16223 raft_consensus.cc:738] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.172501 16223 consensus_queue.cc:260] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.173172 16223 ts_tablet_manager.cc:1428] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250905 07:50:45.173453 16223 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.178390 16223 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:45.178766 16223 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.178977 16223 ts_tablet_manager.cc:1397] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20250905 07:50:45.179555 16223 raft_consensus.cc:357] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.179697 16223 raft_consensus.cc:738] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.179942 16223 consensus_queue.cc:260] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.180326 16223 ts_tablet_manager.cc:1428] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250905 07:50:45.180533 16223 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.183907 16223 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:45.184214 16223 tablet_bootstrap.cc:492] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.184458 16223 ts_tablet_manager.cc:1397] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250905 07:50:45.184973 16223 raft_consensus.cc:357] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.185103 16223 raft_consensus.cc:738] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.185302 16223 consensus_queue.cc:260] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.185663 16223 ts_tablet_manager.cc:1428] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:45.185858 16223 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.186276 9109 rpc_server.cc:307] RPC server started. Bound to: 127.8.229.65:44917
I20250905 07:50:45.186280 16285 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.229.65:44917 every 8 connection(s)
I20250905 07:50:45.191800 16223 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:45.192209 16223 tablet_bootstrap.cc:492] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.192459 16223 ts_tablet_manager.cc:1397] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20250905 07:50:45.193157 16223 raft_consensus.cc:357] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:45.193327 16223 raft_consensus.cc:738] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.193567 16223 consensus_queue.cc:260] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:45.194063 16223 ts_tablet_manager.cc:1428] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:50:45.194306 16223 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.194696 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:45.195185 16286 heartbeater.cc:344] Connected to a master server at 127.8.229.126:34109
I20250905 07:50:45.195343 16286 heartbeater.cc:461] Registering TS with master...
I20250905 07:50:45.195807 16286 heartbeater.cc:507] Master 127.8.229.126:34109 requested a full tablet report, sending...
W20250905 07:50:45.195889 15733 catalog_manager.cc:4908] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): delete failed for tablet 4ab5d983123841ee8cdf655cc05fe09d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ab5d983123841ee8cdf655cc05fe09d already in progress: opening tablet
I20250905 07:50:45.197203 15744 ts_manager.cc:194] Re-registered known tserver with Master: 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917)
I20250905 07:50:45.198870 16223 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:45.199237 16223 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.199470 16223 ts_tablet_manager.cc:1397] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20250905 07:50:45.200157 16223 raft_consensus.cc:357] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.200222 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:45.200323 16223 raft_consensus.cc:738] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.200583 15744 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40020
I20250905 07:50:45.200552 16223 consensus_queue.cc:260] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
W20250905 07:50:45.200780 15733 catalog_manager.cc:4908] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): delete failed for tablet 4ab5d983123841ee8cdf655cc05fe09d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ab5d983123841ee8cdf655cc05fe09d already in progress: opening tablet
I20250905 07:50:45.201026 16223 ts_tablet_manager.cc:1428] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:45.201253 16223 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Bootstrap starting.
I20250905 07:50:45.201726 16286 heartbeater.cc:499] Master 127.8.229.126:34109 was elected leader, sending a full tablet report...
I20250905 07:50:45.205180 16223 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:45.205473 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:45.205575 16223 tablet_bootstrap.cc:492] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Bootstrap complete.
I20250905 07:50:45.205793 16223 ts_tablet_manager.cc:1397] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
W20250905 07:50:45.205998 15733 catalog_manager.cc:4908] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): delete failed for tablet 4ab5d983123841ee8cdf655cc05fe09d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ab5d983123841ee8cdf655cc05fe09d already in progress: opening tablet
I20250905 07:50:45.206358 16223 raft_consensus.cc:357] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.206482 16223 raft_consensus.cc:738] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8673871edcb144dc863da6e6fb876fad, State: Initialized, Role: FOLLOWER
I20250905 07:50:45.206687 16223 consensus_queue.cc:260] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.207088 16223 ts_tablet_manager.cc:1428] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250905 07:50:45.208932 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet 4ab5d983123841ee8cdf655cc05fe09d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:45.209136 16291 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:45.209290 16291 raft_consensus.cc:2241] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.209439 16291 raft_consensus.cc:2270] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.210156 16291 ts_tablet_manager.cc:1905] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.214344 16291 ts_tablet_manager.cc:1918] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.214440 16291 log.cc:1199] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/4ab5d983123841ee8cdf655cc05fe09d
I20250905 07:50:45.214699 16291 ts_tablet_manager.cc:1939] T 4ab5d983123841ee8cdf655cc05fe09d P 8673871edcb144dc863da6e6fb876fad: Deleting consensus metadata
I20250905 07:50:45.215466 15733 catalog_manager.cc:4928] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): tablet 4ab5d983123841ee8cdf655cc05fe09d (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.227277 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet 0df306fbdd3a4d1aa71e2bc678d35465 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:45.227581 16291 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:45.227766 16291 raft_consensus.cc:2241] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.227918 16291 raft_consensus.cc:2270] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.228593 16291 ts_tablet_manager.cc:1905] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.233085 16291 ts_tablet_manager.cc:1918] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250905 07:50:45.233175 16291 log.cc:1199] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/0df306fbdd3a4d1aa71e2bc678d35465
I20250905 07:50:45.233459 16291 ts_tablet_manager.cc:1939] T 0df306fbdd3a4d1aa71e2bc678d35465 P 8673871edcb144dc863da6e6fb876fad: Deleting consensus metadata
I20250905 07:50:45.234300 15733 catalog_manager.cc:4928] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): tablet 0df306fbdd3a4d1aa71e2bc678d35465 (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.254033 16118 raft_consensus.cc:491] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.254225 16118 raft_consensus.cc:513] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.254828 16118 leader_election.cc:290] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:45.255395 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:45.259656 16260 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:45.259855 16260 raft_consensus.cc:2466] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:45.260196 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet a0fba91ca8d143f39bd2d3cf76cfc13f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:45.260308 15889 leader_election.cc:304] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters: a6f256ebcfff460ea87eaa92563a9231
I20250905 07:50:45.260485 16291 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:45.260619 16118 raft_consensus.cc:2802] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:45.260711 16118 raft_consensus.cc:491] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.260716 16291 raft_consensus.cc:2241] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.260850 16118 raft_consensus.cc:3058] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.260960 16291 raft_consensus.cc:2270] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.261746 16291 ts_tablet_manager.cc:1905] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.263697 16118 raft_consensus.cc:513] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.264323 16118 leader_election.cc:290] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 election: Requested vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:45.264664 16260 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad"
I20250905 07:50:45.264621 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:45.264845 16260 raft_consensus.cc:3058] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.265594 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet 257f5535e785474f9d4f2eaf93040ba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-09-05 07:50:45 UTC) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:45.267167 16260 raft_consensus.cc:2466] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 2.
I20250905 07:50:45.267381 16291 ts_tablet_manager.cc:1918] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250905 07:50:45.267537 16291 log.cc:1199] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/a0fba91ca8d143f39bd2d3cf76cfc13f
I20250905 07:50:45.267518 15889 leader_election.cc:304] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters: a6f256ebcfff460ea87eaa92563a9231
I20250905 07:50:45.267804 16118 raft_consensus.cc:2802] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:45.267932 16291 ts_tablet_manager.cc:1939] T a0fba91ca8d143f39bd2d3cf76cfc13f P 8673871edcb144dc863da6e6fb876fad: Deleting consensus metadata
I20250905 07:50:45.267982 16118 raft_consensus.cc:695] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Becoming Leader. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Running, Role: LEADER
I20250905 07:50:45.268289 16118 consensus_queue.cc:237] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.268903 16291 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:45.268970 15733 catalog_manager.cc:4928] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): tablet a0fba91ca8d143f39bd2d3cf76cfc13f (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.269119 16291 raft_consensus.cc:2241] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:45.269299 16291 pending_rounds.cc:70] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Trying to abort 1 pending ops.
I20250905 07:50:45.269433 16291 pending_rounds.cc:77] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196912204271247360 op_type: NO_OP noop_request { }
I20250905 07:50:45.269588 16291 raft_consensus.cc:2887] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20250905 07:50:45.269721 16291 raft_consensus.cc:2270] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:45.270406 16291 ts_tablet_manager.cc:1905] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250905 07:50:45.270834 15744 catalog_manager.cc:5582] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: term changed from 1 to 2, leader changed from 8673871edcb144dc863da6e6fb876fad (127.8.229.65) to 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66). New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:45.275084 16291 ts_tablet_manager.cc:1918] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250905 07:50:45.275200 16291 log.cc:1199] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/257f5535e785474f9d4f2eaf93040ba0
I20250905 07:50:45.275574 16291 ts_tablet_manager.cc:1939] T 257f5535e785474f9d4f2eaf93040ba0 P 8673871edcb144dc863da6e6fb876fad: Deleting consensus metadata
I20250905 07:50:45.276381 15733 catalog_manager.cc:4928] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): tablet 257f5535e785474f9d4f2eaf93040ba0 (table dugtrio [id=1504cfbc6e914551b346d3724cb098b1]) successfully deleted
I20250905 07:50:45.313587 16118 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.313799 16118 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:45.314457 16118 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795), 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917)
I20250905 07:50:45.314666 16135 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.314847 16135 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:45.314985 16260 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:45.314990 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231" is_pre_election: true
I20250905 07:50:45.315246 15997 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:45.315253 16260 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 1.
I20250905 07:50:45.315543 16135 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:45.315822 15889 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters:
I20250905 07:50:45.315922 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:45.316165 15922 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6f256ebcfff460ea87eaa92563a9231 in term 1.
I20250905 07:50:45.316553 16118 raft_consensus.cc:2802] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:45.316529 15960 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:45.316712 16118 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.316834 16118 raft_consensus.cc:3058] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.316848 16135 raft_consensus.cc:2802] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:45.316977 16135 raft_consensus.cc:491] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.317104 16135 raft_consensus.cc:3058] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.319280 16135 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:45.319456 16118 raft_consensus.cc:513] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:45.320047 16118 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 election: Requested vote from peers a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795), 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917)
I20250905 07:50:45.320281 16135 leader_election.cc:290] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 election: Requested vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:45.320341 15997 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a6f256ebcfff460ea87eaa92563a9231"
I20250905 07:50:45.320485 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
I20250905 07:50:45.320562 15997 raft_consensus.cc:2391] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 448a1f2f153145c0aa61c7cf3ed83d27 in current term 2: Already voted for candidate a6f256ebcfff460ea87eaa92563a9231 in this term.
I20250905 07:50:45.320685 15922 raft_consensus.cc:2391] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a6f256ebcfff460ea87eaa92563a9231 in current term 2: Already voted for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in this term.
I20250905 07:50:45.320977 16260 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad"
I20250905 07:50:45.321151 16260 raft_consensus.cc:3058] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.322165 16259 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:45.322285 16258 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2df7d42ec18145c1b84430ee2e02ed00" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad"
I20250905 07:50:45.322876 15963 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6f256ebcfff460ea87eaa92563a9231; no voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad
I20250905 07:50:45.323161 16135 raft_consensus.cc:2747] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250905 07:50:45.323544 16260 raft_consensus.cc:2466] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in term 2.
I20250905 07:50:45.323834 15889 leader_election.cc:304] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad; no voters: a6f256ebcfff460ea87eaa92563a9231
I20250905 07:50:45.324079 16118 raft_consensus.cc:2802] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:45.324208 16118 raft_consensus.cc:695] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Becoming Leader. State: Replica: 448a1f2f153145c0aa61c7cf3ed83d27, State: Running, Role: LEADER
I20250905 07:50:45.324470 16118 consensus_queue.cc:237] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
I20250905 07:50:45.326776 15744 catalog_manager.cc:5582] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: term changed from 1 to 2, leader changed from 8673871edcb144dc863da6e6fb876fad (127.8.229.65) to 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66). New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:45.336418 16135 raft_consensus.cc:491] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.336614 16135 raft_consensus.cc:513] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.337209 16135 leader_election.cc:290] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:45.337648 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:45.337652 16260 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:45.337914 16260 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6f256ebcfff460ea87eaa92563a9231 in term 1.
I20250905 07:50:45.337915 15922 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6f256ebcfff460ea87eaa92563a9231 in term 1.
I20250905 07:50:45.338361 15963 leader_election.cc:304] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:45.338629 16135 raft_consensus.cc:2802] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250905 07:50:45.338709 16135 raft_consensus.cc:491] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.338814 16135 raft_consensus.cc:3058] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.340772 16135 raft_consensus.cc:513] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.341269 16135 leader_election.cc:290] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 election: Requested vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:45.341645 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27"
I20250905 07:50:45.341666 16260 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "db5db38b839d4ff6b71ba9d9735fa73a" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad"
I20250905 07:50:45.341835 16260 raft_consensus.cc:3058] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.341836 15922 raft_consensus.cc:3058] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.343796 16260 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6f256ebcfff460ea87eaa92563a9231 in term 2.
I20250905 07:50:45.343792 15922 raft_consensus.cc:2466] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6f256ebcfff460ea87eaa92563a9231 in term 2.
I20250905 07:50:45.344215 15963 leader_election.cc:304] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8673871edcb144dc863da6e6fb876fad, a6f256ebcfff460ea87eaa92563a9231; no voters:
I20250905 07:50:45.344449 16135 raft_consensus.cc:2802] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Leader election won for term 2
I20250905 07:50:45.344810 16135 raft_consensus.cc:695] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 LEADER]: Becoming Leader. State: Replica: a6f256ebcfff460ea87eaa92563a9231, State: Running, Role: LEADER
I20250905 07:50:45.345239 16135 consensus_queue.cc:237] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.347522 15744 catalog_manager.cc:5582] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 reported cstate change: term changed from 1 to 2, leader changed from 8673871edcb144dc863da6e6fb876fad (127.8.229.65) to a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67). New cstate: current_term: 2 leader_uuid: "a6f256ebcfff460ea87eaa92563a9231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: HEALTHY } } }
I20250905 07:50:45.401687 16135 raft_consensus.cc:491] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8673871edcb144dc863da6e6fb876fad)
I20250905 07:50:45.401896 16135 raft_consensus.cc:513] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.402603 16135 leader_election.cc:290] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917), 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:45.403081 15922 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" is_pre_election: true
I20250905 07:50:45.403079 16260 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c894b6ee25a4fac8290c1edc8243572" candidate_uuid: "a6f256ebcfff460ea87eaa92563a9231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8673871edcb144dc863da6e6fb876fad" is_pre_election: true
I20250905 07:50:45.403399 16260 raft_consensus.cc:2391] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6f256ebcfff460ea87eaa92563a9231 in current term 2: Already voted for candidate 448a1f2f153145c0aa61c7cf3ed83d27 in this term.
I20250905 07:50:45.403821 15963 leader_election.cc:304] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6f256ebcfff460ea87eaa92563a9231; no voters: 448a1f2f153145c0aa61c7cf3ed83d27, 8673871edcb144dc863da6e6fb876fad
I20250905 07:50:45.404083 16135 raft_consensus.cc:3058] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.406035 16135 raft_consensus.cc:2747] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250905 07:50:45.441195 16260 raft_consensus.cc:3058] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 1 FOLLOWER]: Advancing to term 2
I20250905 07:50:45.443507 16260 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.444067 16118 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:50:45.447737 15997 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.448490 16100 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250905 07:50:45.533876 15794 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20250905 07:50:45.535250 15794 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 2df7d42ec18145c1b84430ee2e02ed00, TS a6f256ebcfff460ea87eaa92563a9231 -> TS 742142751ce0432db4d94405382c6dc6 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
W20250905 07:50:45.535329 15794 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 2df7d42ec18145c1b84430ee2e02ed00, TS a6f256ebcfff460ea87eaa92563a9231 -> TS 742142751ce0432db4d94405382c6dc6 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } }
W20250905 07:50:45.536545 15794 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 135d618a63ea48448bf95be21e340ce5, TS 8673871edcb144dc863da6e6fb876fad -> TS 742142751ce0432db4d94405382c6dc6 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
W20250905 07:50:45.536619 15794 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 135d618a63ea48448bf95be21e340ce5, TS 8673871edcb144dc863da6e6fb876fad -> TS 742142751ce0432db4d94405382c6dc6 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
W20250905 07:50:45.537847 15794 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet db5db38b839d4ff6b71ba9d9735fa73a, TS 448a1f2f153145c0aa61c7cf3ed83d27 -> TS 742142751ce0432db4d94405382c6dc6 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
W20250905 07:50:45.537927 15794 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet db5db38b839d4ff6b71ba9d9735fa73a, TS 448a1f2f153145c0aa61c7cf3ed83d27 -> TS 742142751ce0432db4d94405382c6dc6 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } }
I20250905 07:50:45.745553 15997 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.746156 16100 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:50:45.749326 16260 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.749899 16306 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:50:45.810338 15922 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.810941 16135 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:50:45.814121 16260 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.814673 16135 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:50:45.851588 15997 raft_consensus.cc:1273] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.852100 16306 consensus_queue.cc:1035] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:50:45.855147 16260 raft_consensus.cc:1273] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250905 07:50:45.855780 16100 consensus_queue.cc:1035] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250905 07:50:46.123029 16205 heartbeater.cc:499] Master 127.8.229.126:34109 was elected leader, sending a full tablet report...
W20250905 07:50:46.540190 15794 auto_rebalancer.cc:249] could not retrieve auto-rebalancing replica moves: Not found: table 60f301b140254606998f8e2e5c8f4c51: could not find any suitable replica to move from server 448a1f2f153145c0aa61c7cf3ed83d27 to server 742142751ce0432db4d94405382c6dc6
I20250905 07:50:47.544118 15922 consensus_queue.cc:237] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:47.546290 16260 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:47.546916 16118 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:47.547534 15997 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:47.547983 16118 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:47.549461 16314 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.551514 15997 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.551851 16260 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.555406 15744 catalog_manager.cc:5582] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250905 07:50:47.556819 15887 consensus_peers.cc:489] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 -> Peer 742142751ce0432db4d94405382c6dc6 (127.8.229.68:33779): Couldn't send request to peer 742142751ce0432db4d94405382c6dc6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c894b6ee25a4fac8290c1edc8243572. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:47.557389 15922 consensus_queue.cc:237] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:47.559471 16260 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20250905 07:50:47.559587 15887 consensus_peers.cc:489] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 -> Peer 742142751ce0432db4d94405382c6dc6 (127.8.229.68:33779): Couldn't send request to peer 742142751ce0432db4d94405382c6dc6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 135d618a63ea48448bf95be21e340ce5. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:47.560037 16314 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:47.560158 15997 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:47.560822 16118 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:47.568670 16118 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.570703 15996 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.572456 16260 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.573961 15746 catalog_manager.cc:5582] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250905 07:50:47.582520 15996 consensus_queue.cc:237] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:47.585018 16260 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:47.585593 16312 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:47.586817 15922 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250905 07:50:47.587285 16331 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250905 07:50:47.588357 16312 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.588873 16260 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
W20250905 07:50:47.594146 15961 consensus_peers.cc:489] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 -> Peer 742142751ce0432db4d94405382c6dc6 (127.8.229.68:33779): Couldn't send request to peer 742142751ce0432db4d94405382c6dc6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db5db38b839d4ff6b71ba9d9735fa73a. This is attempt 1: this message will repeat every 5th retry.
I20250905 07:50:47.594172 15746 catalog_manager.cc:5582] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad reported cstate change: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New cstate: current_term: 2 leader_uuid: "a6f256ebcfff460ea87eaa92563a9231" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.605162 15922 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 742142751ce0432db4d94405382c6dc6 (127.8.229.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } } }
I20250905 07:50:47.995823 16338 ts_tablet_manager.cc:927] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: Initiating tablet copy from peer a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795)
I20250905 07:50:47.996488 16338 tablet_copy_client.cc:323] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.67:37795
I20250905 07:50:48.002388 16007 tablet_copy_service.cc:140] P a6f256ebcfff460ea87eaa92563a9231: Received BeginTabletCopySession request for tablet db5db38b839d4ff6b71ba9d9735fa73a from peer 742142751ce0432db4d94405382c6dc6 ({username='slave'} at 127.0.0.1:52356)
I20250905 07:50:48.002542 16007 tablet_copy_service.cc:161] P a6f256ebcfff460ea87eaa92563a9231: Beginning new tablet copy session on tablet db5db38b839d4ff6b71ba9d9735fa73a from peer 742142751ce0432db4d94405382c6dc6 at {username='slave'} at 127.0.0.1:52356: session id = 742142751ce0432db4d94405382c6dc6-db5db38b839d4ff6b71ba9d9735fa73a
I20250905 07:50:48.005543 16007 tablet_copy_source_session.cc:215] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:48.006695 16338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db5db38b839d4ff6b71ba9d9735fa73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:48.011453 16338 tablet_copy_client.cc:806] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:48.011690 16338 tablet_copy_client.cc:670] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:48.013084 16338 tablet_copy_client.cc:538] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:48.015662 16338 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: Bootstrap starting.
I20250905 07:50:48.021560 16338 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:48.021903 16338 tablet_bootstrap.cc:492] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: Bootstrap complete.
I20250905 07:50:48.022107 16338 ts_tablet_manager.cc:1397] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250905 07:50:48.022802 16338 raft_consensus.cc:357] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:48.023033 16338 raft_consensus.cc:738] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 742142751ce0432db4d94405382c6dc6, State: Initialized, Role: LEARNER
I20250905 07:50:48.023260 16338 consensus_queue.cc:260] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:48.024488 16338 ts_tablet_manager.cc:1428] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20250905 07:50:48.025148 16007 tablet_copy_service.cc:342] P a6f256ebcfff460ea87eaa92563a9231: Request end of tablet copy session 742142751ce0432db4d94405382c6dc6-db5db38b839d4ff6b71ba9d9735fa73a received from {username='slave'} at 127.0.0.1:52356
I20250905 07:50:48.025240 16007 tablet_copy_service.cc:434] P a6f256ebcfff460ea87eaa92563a9231: ending tablet copy session 742142751ce0432db4d94405382c6dc6-db5db38b839d4ff6b71ba9d9735fa73a on tablet db5db38b839d4ff6b71ba9d9735fa73a with peer 742142751ce0432db4d94405382c6dc6
I20250905 07:50:48.028434 16338 ts_tablet_manager.cc:927] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: Initiating tablet copy from peer 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:48.028951 16338 tablet_copy_client.cc:323] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:46761
I20250905 07:50:48.035574 15933 tablet_copy_service.cc:140] P 448a1f2f153145c0aa61c7cf3ed83d27: Received BeginTabletCopySession request for tablet 2c894b6ee25a4fac8290c1edc8243572 from peer 742142751ce0432db4d94405382c6dc6 ({username='slave'} at 127.0.0.1:41252)
I20250905 07:50:48.035698 15933 tablet_copy_service.cc:161] P 448a1f2f153145c0aa61c7cf3ed83d27: Beginning new tablet copy session on tablet 2c894b6ee25a4fac8290c1edc8243572 from peer 742142751ce0432db4d94405382c6dc6 at {username='slave'} at 127.0.0.1:41252: session id = 742142751ce0432db4d94405382c6dc6-2c894b6ee25a4fac8290c1edc8243572
I20250905 07:50:48.038010 15933 tablet_copy_source_session.cc:215] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:48.038991 16338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c894b6ee25a4fac8290c1edc8243572. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:48.043831 16338 tablet_copy_client.cc:806] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:48.044029 16338 tablet_copy_client.cc:670] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:48.045248 16338 tablet_copy_client.cc:538] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:48.047991 16338 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: Bootstrap starting.
I20250905 07:50:48.053412 16338 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:48.053828 16338 tablet_bootstrap.cc:492] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: Bootstrap complete.
I20250905 07:50:48.054073 16338 ts_tablet_manager.cc:1397] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20250905 07:50:48.054785 16338 raft_consensus.cc:357] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:48.055049 16338 raft_consensus.cc:738] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 742142751ce0432db4d94405382c6dc6, State: Initialized, Role: LEARNER
I20250905 07:50:48.055245 16338 consensus_queue.cc:260] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:48.055902 16338 ts_tablet_manager.cc:1428] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20250905 07:50:48.056907 15933 tablet_copy_service.cc:342] P 448a1f2f153145c0aa61c7cf3ed83d27: Request end of tablet copy session 742142751ce0432db4d94405382c6dc6-2c894b6ee25a4fac8290c1edc8243572 received from {username='slave'} at 127.0.0.1:41252
I20250905 07:50:48.057003 15933 tablet_copy_service.cc:434] P 448a1f2f153145c0aa61c7cf3ed83d27: ending tablet copy session 742142751ce0432db4d94405382c6dc6-2c894b6ee25a4fac8290c1edc8243572 on tablet 2c894b6ee25a4fac8290c1edc8243572 with peer 742142751ce0432db4d94405382c6dc6
I20250905 07:50:48.076247 16338 ts_tablet_manager.cc:927] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: Initiating tablet copy from peer 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761)
I20250905 07:50:48.076916 16338 tablet_copy_client.cc:323] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: tablet copy: Beginning tablet copy session from remote peer at address 127.8.229.66:46761
I20250905 07:50:48.077574 15933 tablet_copy_service.cc:140] P 448a1f2f153145c0aa61c7cf3ed83d27: Received BeginTabletCopySession request for tablet 135d618a63ea48448bf95be21e340ce5 from peer 742142751ce0432db4d94405382c6dc6 ({username='slave'} at 127.0.0.1:41252)
I20250905 07:50:48.077702 15933 tablet_copy_service.cc:161] P 448a1f2f153145c0aa61c7cf3ed83d27: Beginning new tablet copy session on tablet 135d618a63ea48448bf95be21e340ce5 from peer 742142751ce0432db4d94405382c6dc6 at {username='slave'} at 127.0.0.1:41252: session id = 742142751ce0432db4d94405382c6dc6-135d618a63ea48448bf95be21e340ce5
I20250905 07:50:48.080102 15933 tablet_copy_source_session.cc:215] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: Tablet Copy: opened 0 blocks and 1 log segments
I20250905 07:50:48.081192 16338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135d618a63ea48448bf95be21e340ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250905 07:50:48.086127 16338 tablet_copy_client.cc:806] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: tablet copy: Starting download of 0 data blocks...
I20250905 07:50:48.086395 16338 tablet_copy_client.cc:670] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: tablet copy: Starting download of 1 WAL segments...
I20250905 07:50:48.087798 16338 tablet_copy_client.cc:538] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250905 07:50:48.090350 16338 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: Bootstrap starting.
I20250905 07:50:48.096031 16338 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250905 07:50:48.096462 16338 tablet_bootstrap.cc:492] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: Bootstrap complete.
I20250905 07:50:48.096668 16338 ts_tablet_manager.cc:1397] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: Time spent bootstrapping tablet: real 0.006s user 0.007s sys 0.001s
I20250905 07:50:48.097352 16338 raft_consensus.cc:357] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:48.097513 16338 raft_consensus.cc:738] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 742142751ce0432db4d94405382c6dc6, State: Initialized, Role: LEARNER
I20250905 07:50:48.097733 16338 consensus_queue.cc:260] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: NON_VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: true } }
I20250905 07:50:48.098248 16338 ts_tablet_manager.cc:1428] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250905 07:50:48.098834 15933 tablet_copy_service.cc:342] P 448a1f2f153145c0aa61c7cf3ed83d27: Request end of tablet copy session 742142751ce0432db4d94405382c6dc6-135d618a63ea48448bf95be21e340ce5 received from {username='slave'} at 127.0.0.1:41252
I20250905 07:50:48.098939 15933 tablet_copy_service.cc:434] P 448a1f2f153145c0aa61c7cf3ed83d27: ending tablet copy session 742142751ce0432db4d94405382c6dc6-135d618a63ea48448bf95be21e340ce5 on tablet 135d618a63ea48448bf95be21e340ce5 with peer 742142751ce0432db4d94405382c6dc6
I20250905 07:50:48.439749 16180 raft_consensus.cc:1215] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250905 07:50:48.552186 16180 raft_consensus.cc:1215] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250905 07:50:48.695060 16180 raft_consensus.cc:1215] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250905 07:50:49.000129 16351 raft_consensus.cc:1062] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: attempting to promote NON_VOTER 742142751ce0432db4d94405382c6dc6 to VOTER
I20250905 07:50:49.000913 16351 consensus_queue.cc:237] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } }
I20250905 07:50:49.002921 15923 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.003010 16260 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.003556 16180 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.003507 16353 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.003773 16350 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.004416 16350 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.007193 16353 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.007894 16180 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.010658 16260 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.014158 15743 catalog_manager.cc:5582] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 reported cstate change: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a6f256ebcfff460ea87eaa92563a9231" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:49.022899 15922 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } attrs { replace: true } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.027612 15996 consensus_queue.cc:237] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } }
I20250905 07:50:49.029284 16180 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:49.029758 16350 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250905 07:50:49.029896 16260 raft_consensus.cc:1273] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer a6f256ebcfff460ea87eaa92563a9231: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:49.030727 16348 raft_consensus.cc:1062] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: attempting to promote NON_VOTER 742142751ce0432db4d94405382c6dc6 to VOTER
I20250905 07:50:49.031492 16348 consensus_queue.cc:237] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } }
I20250905 07:50:49.031508 16353 consensus_queue.cc:1035] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250905 07:50:49.033483 16337 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.033998 16180 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.035610 16258 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.035732 16179 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.036079 16349 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.036561 16118 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.038738 15731 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet db5db38b839d4ff6b71ba9d9735fa73a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:49.039289 15996 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.039839 15746 catalog_manager.cc:5582] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 reported cstate change: config changed from index 4 to 5, VOTER 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66) evicted. New cstate: current_term: 2 leader_uuid: "a6f256ebcfff460ea87eaa92563a9231" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:49.040016 16348 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.048708 16260 raft_consensus.cc:2953] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.063074 15913 tablet_service.cc:1515] Processing DeleteTablet for tablet db5db38b839d4ff6b71ba9d9735fa73a with delete_type TABLET_DATA_TOMBSTONED (TS 448a1f2f153145c0aa61c7cf3ed83d27 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41204
I20250905 07:50:49.070461 16348 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.072240 16260 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.073119 15996 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.077559 16179 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.078249 15744 catalog_manager.cc:5582] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 reported cstate change: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } attrs { replace: true } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.082188 16367 tablet_replica.cc:331] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:49.082412 16367 raft_consensus.cc:2241] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:49.082998 16367 raft_consensus.cc:2270] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:49.086696 15922 consensus_queue.cc:237] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } }
I20250905 07:50:49.088352 16367 ts_tablet_manager.cc:1905] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:50:49.088758 16179 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:49.089262 16349 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250905 07:50:49.090862 16260 raft_consensus.cc:1273] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:49.091475 16349 consensus_queue.cc:1035] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250905 07:50:49.092538 16363 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.094374 16258 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.095664 16367 ts_tablet_manager.cc:1918] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250905 07:50:49.095762 16367 log.cc:1199] T db5db38b839d4ff6b71ba9d9735fa73a P 448a1f2f153145c0aa61c7cf3ed83d27: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-1-root/wals/db5db38b839d4ff6b71ba9d9735fa73a
I20250905 07:50:49.096480 15730 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 2c894b6ee25a4fac8290c1edc8243572 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:49.096697 15730 catalog_manager.cc:4928] TS 448a1f2f153145c0aa61c7cf3ed83d27 (127.8.229.66:46761): tablet db5db38b839d4ff6b71ba9d9735fa73a (table test-workload [id=60f301b140254606998f8e2e5c8f4c51]) successfully deleted
I20250905 07:50:49.099742 15746 catalog_manager.cc:5582] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad reported cstate change: config changed from index 4 to 5, VOTER a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67) evicted. New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.100891 16179 raft_consensus.cc:2953] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.103590 15985 tablet_service.cc:1515] Processing DeleteTablet for tablet 2c894b6ee25a4fac8290c1edc8243572 with delete_type TABLET_DATA_TOMBSTONED (TS a6f256ebcfff460ea87eaa92563a9231 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52330
I20250905 07:50:49.108495 16368 tablet_replica.cc:331] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:49.108724 16368 raft_consensus.cc:2241] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:49.109530 16368 raft_consensus.cc:2270] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:49.111016 16368 ts_tablet_manager.cc:1905] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:50:49.113131 16349 raft_consensus.cc:1062] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: attempting to promote NON_VOTER 742142751ce0432db4d94405382c6dc6 to VOTER
I20250905 07:50:49.113754 16349 consensus_queue.cc:237] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } }
I20250905 07:50:49.116043 15996 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.116523 16363 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.116588 16179 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 LEARNER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.117437 16260 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250905 07:50:49.117930 16363 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.118968 16348 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250905 07:50:49.130511 16368 ts_tablet_manager.cc:1918] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250905 07:50:49.135488 16368 log.cc:1199] T 2c894b6ee25a4fac8290c1edc8243572 P a6f256ebcfff460ea87eaa92563a9231: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-2-root/wals/2c894b6ee25a4fac8290c1edc8243572
I20250905 07:50:49.136124 16348 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.136613 16179 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.139662 15731 catalog_manager.cc:4928] TS a6f256ebcfff460ea87eaa92563a9231 (127.8.229.67:37795): tablet 2c894b6ee25a4fac8290c1edc8243572 (table test-workload [id=60f301b140254606998f8e2e5c8f4c51]) successfully deleted
I20250905 07:50:49.140023 15996 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.140729 15743 catalog_manager.cc:5582] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 reported cstate change: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.142771 16260 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 742142751ce0432db4d94405382c6dc6 (127.8.229.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8673871edcb144dc863da6e6fb876fad" member_type: VOTER last_known_addr { host: "127.8.229.65" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.146791 15922 consensus_queue.cc:237] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } }
I20250905 07:50:49.149587 15996 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:49.150238 16349 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250905 07:50:49.150799 16179 raft_consensus.cc:1273] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Refusing update from remote peer 448a1f2f153145c0aa61c7cf3ed83d27: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250905 07:50:49.151213 16348 consensus_queue.cc:1035] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250905 07:50:49.153288 16349 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8673871edcb144dc863da6e6fb876fad (127.8.229.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.153786 16179 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8673871edcb144dc863da6e6fb876fad (127.8.229.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.155972 15996 raft_consensus.cc:2953] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8673871edcb144dc863da6e6fb876fad (127.8.229.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } } }
I20250905 07:50:49.156993 15730 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 135d618a63ea48448bf95be21e340ce5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250905 07:50:49.158354 15746 catalog_manager.cc:5582] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 reported cstate change: config changed from index 4 to 5, VOTER 8673871edcb144dc863da6e6fb876fad (127.8.229.65) evicted. New cstate: current_term: 2 leader_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "448a1f2f153145c0aa61c7cf3ed83d27" member_type: VOTER last_known_addr { host: "127.8.229.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6f256ebcfff460ea87eaa92563a9231" member_type: VOTER last_known_addr { host: "127.8.229.67" port: 37795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "742142751ce0432db4d94405382c6dc6" member_type: VOTER last_known_addr { host: "127.8.229.68" port: 33779 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250905 07:50:49.162045 16249 tablet_service.cc:1515] Processing DeleteTablet for tablet 135d618a63ea48448bf95be21e340ce5 with delete_type TABLET_DATA_TOMBSTONED (TS 8673871edcb144dc863da6e6fb876fad not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:58276
I20250905 07:50:49.162833 16373 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:49.163046 16373 raft_consensus.cc:2241] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:49.163525 16373 raft_consensus.cc:2270] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:49.164742 16373 ts_tablet_manager.cc:1905] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250905 07:50:49.172017 16373 ts_tablet_manager.cc:1918] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250905 07:50:49.172148 16373 log.cc:1199] T 135d618a63ea48448bf95be21e340ce5 P 8673871edcb144dc863da6e6fb876fad: Deleting WAL directory at /tmp/dist-test-task52Y2Rr/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1757058461885731-9109-0/minicluster-data/ts-0-root/wals/135d618a63ea48448bf95be21e340ce5
I20250905 07:50:49.172930 15733 catalog_manager.cc:4928] TS 8673871edcb144dc863da6e6fb876fad (127.8.229.65:44917): tablet 135d618a63ea48448bf95be21e340ce5 (table test-workload [id=60f301b140254606998f8e2e5c8f4c51]) successfully deleted
I20250905 07:50:51.737735 9109 tablet_server.cc:178] TabletServer@127.8.229.65:44917 shutting down...
I20250905 07:50:51.759518 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:51.759800 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:51.760039 9109 raft_consensus.cc:2241] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.760362 9109 raft_consensus.cc:2270] T 2c894b6ee25a4fac8290c1edc8243572 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.761092 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:51.761241 9109 raft_consensus.cc:2241] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.761505 9109 raft_consensus.cc:2270] T db5db38b839d4ff6b71ba9d9735fa73a P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.762040 9109 tablet_replica.cc:331] stopping tablet replica
I20250905 07:50:51.762168 9109 raft_consensus.cc:2241] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.762331 9109 raft_consensus.cc:2270] T 2df7d42ec18145c1b84430ee2e02ed00 P 8673871edcb144dc863da6e6fb876fad [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.776170 9109 tablet_server.cc:195] TabletServer@127.8.229.65:44917 shutdown complete.
I20250905 07:50:51.788164 9109 tablet_server.cc:178] TabletServer@127.8.229.66:0 shutting down...
I20250905 07:50:51.797014 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:51.797266 9109 tablet_replica.cc:331] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:51.797457 9109 raft_consensus.cc:2241] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:51.797771 9109 raft_consensus.cc:2270] T 2df7d42ec18145c1b84430ee2e02ed00 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.798342 9109 tablet_replica.cc:331] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:51.798473 9109 raft_consensus.cc:2241] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:51.798692 9109 raft_consensus.cc:2270] T 135d618a63ea48448bf95be21e340ce5 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.799098 9109 tablet_replica.cc:331] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27: stopping tablet replica
I20250905 07:50:51.799221 9109 raft_consensus.cc:2241] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:51.799427 9109 raft_consensus.cc:2270] T 2c894b6ee25a4fac8290c1edc8243572 P 448a1f2f153145c0aa61c7cf3ed83d27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.814987 9109 tablet_server.cc:195] TabletServer@127.8.229.66:0 shutdown complete.
I20250905 07:50:51.819248 9109 tablet_server.cc:178] TabletServer@127.8.229.67:0 shutting down...
I20250905 07:50:51.828403 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:51.828621 9109 tablet_replica.cc:331] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:51.828788 9109 raft_consensus.cc:2241] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.829048 9109 raft_consensus.cc:2270] T 135d618a63ea48448bf95be21e340ce5 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.829589 9109 tablet_replica.cc:331] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:51.829715 9109 raft_consensus.cc:2241] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 LEADER]: Raft consensus shutting down.
I20250905 07:50:51.829988 9109 raft_consensus.cc:2270] T db5db38b839d4ff6b71ba9d9735fa73a P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.830404 9109 tablet_replica.cc:331] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231: stopping tablet replica
I20250905 07:50:51.830529 9109 raft_consensus.cc:2241] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.830649 9109 raft_consensus.cc:2270] T 2df7d42ec18145c1b84430ee2e02ed00 P a6f256ebcfff460ea87eaa92563a9231 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.844497 9109 tablet_server.cc:195] TabletServer@127.8.229.67:0 shutdown complete.
I20250905 07:50:51.848675 9109 tablet_server.cc:178] TabletServer@127.8.229.68:0 shutting down...
I20250905 07:50:51.857890 9109 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250905 07:50:51.858119 9109 tablet_replica.cc:331] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6: stopping tablet replica
I20250905 07:50:51.858309 9109 raft_consensus.cc:2241] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.858562 9109 raft_consensus.cc:2270] T db5db38b839d4ff6b71ba9d9735fa73a P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.859120 9109 tablet_replica.cc:331] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6: stopping tablet replica
I20250905 07:50:51.859251 9109 raft_consensus.cc:2241] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.859453 9109 raft_consensus.cc:2270] T 135d618a63ea48448bf95be21e340ce5 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.859887 9109 tablet_replica.cc:331] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6: stopping tablet replica
I20250905 07:50:51.868170 9109 raft_consensus.cc:2241] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250905 07:50:51.868391 9109 raft_consensus.cc:2270] T 2c894b6ee25a4fac8290c1edc8243572 P 742142751ce0432db4d94405382c6dc6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.881826 9109 tablet_server.cc:195] TabletServer@127.8.229.68:0 shutdown complete.
I20250905 07:50:51.885146 9109 master.cc:561] Master@127.8.229.126:34109 shutting down...
I20250905 07:50:51.892367 9109 raft_consensus.cc:2241] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 1 LEADER]: Raft consensus shutting down.
I20250905 07:50:51.892563 9109 raft_consensus.cc:2270] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250905 07:50:51.892654 9109 tablet_replica.cc:331] T 00000000000000000000000000000000 P d364292a1f1f4200a3de95c3d8ff2088: stopping tablet replica
I20250905 07:50:51.905697 9109 master.cc:583] Master@127.8.229.126:34109 shutdown complete.
[ OK ] AutoRebalancerTest.TestDeletedTables (10446 ms)
[----------] 14 tests from AutoRebalancerTest (189970 ms total)
[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (189970 ms total)
[ PASSED ] 13 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer
1 FAILED TEST
I20250905 07:50:51.918900 9109 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:594: suppressed but not reported on 7 messages since previous log ~6 seconds ago
I20250905 07:50:51.918982 9109 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:542: suppressed but not reported on 7 messages since previous log ~6 seconds ago
I20250905 07:50:51.919034 9109 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:4671: suppressed but not reported on 11 messages since previous log ~6 seconds ago
I20250905 07:50:51.919087 9109 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:5260: suppressed but not reported on 15 messages since previous log ~33 seconds ago
I20250905 07:50:51.919142 9109 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 8 messages since previous log ~175 seconds ago
I20250905 07:50:51.919235 9109 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 50 messages since previous log ~20 seconds ago
I20250905 07:50:51.919282 9109 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 24 messages since previous log ~10 seconds ago