[==========] Running 14 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 14 tests from AutoRebalancerTest
[ RUN ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251210 21:03:37.709893 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:39107,127.21.127.125:34635,127.21.127.124:43885
I20251210 21:03:37.712227 22013 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251210 21:03:37.713582 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:37.724941 22019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:37.724946 22021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:37.724952 22018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:37.725876 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:37.726714 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:37.726867 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:37.726935 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400617726919 us; error 0 us; skew 500 ppm
I20251210 21:03:37.727245 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:37.729880 22013 webserver.cc:492] Webserver started at http://127.21.127.126:36257/ using document root <none> and password file <none>
I20251210 21:03:37.730579 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:37.730669 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:37.731010 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:37.734337 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.740429 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20251210 21:03:37.743745 22026 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:37.745306 22013 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20251210 21:03:37.745457 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.745616 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:37.764284 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:37.765116 22013 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251210 21:03:37.765362 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:37.785110 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:39107
I20251210 21:03:37.785152 22077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:39107 every 8 connection(s)
I20251210 21:03:37.787361 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:37.790484 22081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:37.791306 22080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:37.791551 22078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251210 21:03:37.791795 22083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:37.793146 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:37.793438 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:37.793502 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:37.793576 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400617793536 us; error 0 us; skew 500 ppm
I20251210 21:03:37.793727 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:37.794915 22013 webserver.cc:492] Webserver started at http://127.21.127.125:36335/ using document root <none> and password file <none>
I20251210 21:03:37.795127 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:37.795214 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:37.795346 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:37.796007 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root/instance:
uuid: "75c609eed8924b3188cead6107717180"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.798609 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251210 21:03:37.800349 22088 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:37.800734 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251210 21:03:37.800848 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root
uuid: "75c609eed8924b3188cead6107717180"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.800980 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:37.801367 22078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:37.818331 22078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } has no permanent_uuid. Determining permanent_uuid...
W20251210 21:03:37.819497 22029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.125:34635: connect: Connection refused (error 111)
W20251210 21:03:37.821396 22078 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.127.125:34635: Network error: Client connection negotiation failed: client connection to 127.21.127.125:34635: connect: Connection refused (error 111)
I20251210 21:03:37.824602 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:37.825148 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:37.839520 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.125:34635
I20251210 21:03:37.839591 22142 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.125:34635 every 8 connection(s)
I20251210 21:03:37.841022 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:03:37.841166 22143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251210 21:03:37.844123 22145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:37.844655 22146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:37.845573 22143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } has no permanent_uuid. Determining permanent_uuid...
W20251210 21:03:37.851017 22148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:37.852660 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:37.852977 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:37.853073 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:37.853150 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400617853110 us; error 0 us; skew 500 ppm
I20251210 21:03:37.853318 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:37.853834 22143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:37.854595 22013 webserver.cc:492] Webserver started at http://127.21.127.124:39795/ using document root <none> and password file <none>
I20251210 21:03:37.854893 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:37.855124 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:37.855268 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:37.855964 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root/instance:
uuid: "90192a3a7f2f4d40a8667f08020c9b7e"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.859185 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251210 21:03:37.860952 22143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:37.861347 22155 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:37.861694 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:03:37.861826 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root
uuid: "90192a3a7f2f4d40a8667f08020c9b7e"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.861980 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251210 21:03:37.868911 22143 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.127.124:43885: Network error: Client connection negotiation failed: client connection to 127.21.127.124:43885: connect: Connection refused (error 111)
I20251210 21:03:37.870712 22078 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } attempt: 1
I20251210 21:03:37.876540 22078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:37.877749 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20251210 21:03:37.878165 22078 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.127.124:43885: Network error: Client connection negotiation failed: client connection to 127.21.127.124:43885: connect: Connection refused (error 111)
I20251210 21:03:37.878383 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:37.891732 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.124:43885
I20251210 21:03:37.891788 22207 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.124:43885 every 8 connection(s)
I20251210 21:03:37.892729 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:03:37.893368 22208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:37.895715 22208 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:37.901993 22208 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:37.907162 22208 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:37.916910 22208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: Bootstrap starting.
I20251210 21:03:37.920495 22208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:37.921176 22143 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } attempt: 1
I20251210 21:03:37.921383 22078 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } attempt: 1
I20251210 21:03:37.921954 22208 log.cc:826] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: Log is configured to *not* fsync() on all Append() calls
I20251210 21:03:37.925455 22208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: No bootstrap required, opened a new log
I20251210 21:03:37.931589 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: Bootstrap starting.
I20251210 21:03:37.932669 22208 raft_consensus.cc:359] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:37.932993 22208 raft_consensus.cc:385] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:37.933079 22143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180: Bootstrap starting.
I20251210 21:03:37.933159 22208 raft_consensus.cc:740] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90192a3a7f2f4d40a8667f08020c9b7e, State: Initialized, Role: FOLLOWER
I20251210 21:03:37.933475 22078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:37.933951 22208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:37.934988 22143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:37.934990 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:37.935396 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:37.936544 22208 sys_catalog.cc:565] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:37.936789 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: No bootstrap required, opened a new log
I20251210 21:03:37.937085 22143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180: No bootstrap required, opened a new log
I20251210 21:03:37.937742 22143 raft_consensus.cc:359] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:37.937750 22078 raft_consensus.cc:359] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:37.937937 22143 raft_consensus.cc:385] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:37.937960 22078 raft_consensus.cc:385] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:37.938031 22143 raft_consensus.cc:740] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c609eed8924b3188cead6107717180, State: Initialized, Role: FOLLOWER
I20251210 21:03:37.938069 22078 raft_consensus.cc:740] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c414d56a9c3e45a5bf6d9f4eeabc1b2c, State: Initialized, Role: FOLLOWER
I20251210 21:03:37.938346 22143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:37.938409 22078 consensus_queue.cc:260] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:37.939334 22219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:37.939656 22143 sys_catalog.cc:565] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:37.939674 22219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:37.945509 22220 sys_catalog.cc:455] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:37.945825 22078 sys_catalog.cc:565] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:37.945848 22220 sys_catalog.cc:458] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [sys.catalog]: This master's current role is: FOLLOWER
W20251210 21:03:37.952278 22234 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251210 21:03:37.952440 22234 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251210 21:03:37.955102 22245 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251210 21:03:37.955273 22245 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251210 21:03:37.956810 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20251210 21:03:37.956938 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20251210 21:03:37.957996 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:37.958037 22252 catalog_manager.cc:1568] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251210 21:03:37.958142 22252 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251210 21:03:37.964430 22253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:37.964982 22254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:37.966374 22013 server_base.cc:1047] running on GCE node
W20251210 21:03:37.966526 22256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:37.966949 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:37.967058 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:37.967121 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400617967100 us; error 0 us; skew 500 ppm
I20251210 21:03:37.967326 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:37.968762 22013 webserver.cc:492] Webserver started at http://127.21.127.65:33397/ using document root <none> and password file <none>
I20251210 21:03:37.969081 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:37.969183 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:37.969352 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:37.970230 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "424fa9035a49437da441bcc03f182de4"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.973953 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20251210 21:03:37.976280 22261 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:37.976698 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:03:37.976826 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "424fa9035a49437da441bcc03f182de4"
format_stamp: "Formatted at 2025-12-10 21:03:37 on dist-test-slave-9pq8"
I20251210 21:03:37.976949 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:37.993048 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:37.993736 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:37.994632 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:37.996222 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:37.996332 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:37.996446 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:37.996526 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:38.015841 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:42837
I20251210 21:03:38.015920 22323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:42837 every 8 connection(s)
W20251210 21:03:38.017442 22032 tablet.cc:2378] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251210 21:03:38.030894 22324 heartbeater.cc:344] Connected to a master server at 127.21.127.124:43885
I20251210 21:03:38.031237 22324 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.032560 22324 heartbeater.cc:507] Master 127.21.127.124:43885 requested a full tablet report, sending...
I20251210 21:03:38.034451 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:03:38.035905 22173 ts_manager.cc:194] Registered new tserver with Master: 424fa9035a49437da441bcc03f182de4 (127.21.127.65:42837)
I20251210 21:03:38.037829 22325 heartbeater.cc:344] Connected to a master server at 127.21.127.126:39107
I20251210 21:03:38.038000 22325 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.038632 22325 heartbeater.cc:507] Master 127.21.127.126:39107 requested a full tablet report, sending...
I20251210 21:03:38.040545 22043 ts_manager.cc:194] Registered new tserver with Master: 424fa9035a49437da441bcc03f182de4 (127.21.127.65:42837)
I20251210 21:03:38.041154 22326 heartbeater.cc:344] Connected to a master server at 127.21.127.125:34635
I20251210 21:03:38.041316 22326 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.041913 22326 heartbeater.cc:507] Master 127.21.127.125:34635 requested a full tablet report, sending...
W20251210 21:03:38.042281 22330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:38.042797 22331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:38.043174 22108 ts_manager.cc:194] Registered new tserver with Master: 424fa9035a49437da441bcc03f182de4 (127.21.127.65:42837)
W20251210 21:03:38.051573 22333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:38.051719 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:38.052196 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:38.052275 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:38.052354 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400618052318 us; error 0 us; skew 500 ppm
I20251210 21:03:38.052522 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:38.053913 22013 webserver.cc:492] Webserver started at http://127.21.127.66:42087/ using document root <none> and password file <none>
I20251210 21:03:38.054142 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:38.054241 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:38.054385 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:38.055193 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "e05dda0e81b545aa9021e5d9795ab9cc"
format_stamp: "Formatted at 2025-12-10 21:03:38 on dist-test-slave-9pq8"
I20251210 21:03:38.057981 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251210 21:03:38.059835 22338 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:38.060272 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:38.060386 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "e05dda0e81b545aa9021e5d9795ab9cc"
format_stamp: "Formatted at 2025-12-10 21:03:38 on dist-test-slave-9pq8"
I20251210 21:03:38.060525 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:38.068801 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:38.069460 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:38.070123 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:38.071158 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:38.071239 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:38.071352 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:38.071426 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:38.090837 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:33997
I20251210 21:03:38.090854 22400 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:33997 every 8 connection(s)
I20251210 21:03:38.104321 22401 heartbeater.cc:344] Connected to a master server at 127.21.127.124:43885
I20251210 21:03:38.104532 22401 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.105472 22401 heartbeater.cc:507] Master 127.21.127.124:43885 requested a full tablet report, sending...
I20251210 21:03:38.106858 22173 ts_manager.cc:194] Registered new tserver with Master: e05dda0e81b545aa9021e5d9795ab9cc (127.21.127.66:33997)
I20251210 21:03:38.109381 22402 heartbeater.cc:344] Connected to a master server at 127.21.127.126:39107
I20251210 21:03:38.109575 22402 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.110291 22402 heartbeater.cc:507] Master 127.21.127.126:39107 requested a full tablet report, sending...
I20251210 21:03:38.111663 22043 ts_manager.cc:194] Registered new tserver with Master: e05dda0e81b545aa9021e5d9795ab9cc (127.21.127.66:33997)
I20251210 21:03:38.113313 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:03:38.116066 22403 heartbeater.cc:344] Connected to a master server at 127.21.127.125:34635
I20251210 21:03:38.116262 22403 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.116922 22403 heartbeater.cc:507] Master 127.21.127.125:34635 requested a full tablet report, sending...
W20251210 21:03:38.117471 22407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:38.118134 22108 ts_manager.cc:194] Registered new tserver with Master: e05dda0e81b545aa9021e5d9795ab9cc (127.21.127.66:33997)
W20251210 21:03:38.121032 22408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:38.122517 22410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:38.123390 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:38.123682 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:38.123764 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:38.123826 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400618123806 us; error 0 us; skew 500 ppm
I20251210 21:03:38.123997 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:38.125394 22013 webserver.cc:492] Webserver started at http://127.21.127.67:39623/ using document root <none> and password file <none>
I20251210 21:03:38.125625 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:38.125705 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:38.125852 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:38.126506 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "fecaf19f8664480eaaffe1b3fe30508b"
format_stamp: "Formatted at 2025-12-10 21:03:38 on dist-test-slave-9pq8"
I20251210 21:03:38.129212 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:03:38.131054 22415 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:38.131493 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:38.131619 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "fecaf19f8664480eaaffe1b3fe30508b"
format_stamp: "Formatted at 2025-12-10 21:03:38 on dist-test-slave-9pq8"
I20251210 21:03:38.131752 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:38.147672 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:38.148552 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:38.149327 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:38.150539 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:38.150658 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:38.150766 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:38.150854 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:38.169245 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:33635
I20251210 21:03:38.169279 22477 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:33635 every 8 connection(s)
I20251210 21:03:38.182106 22478 heartbeater.cc:344] Connected to a master server at 127.21.127.124:43885
I20251210 21:03:38.182372 22478 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.183017 22478 heartbeater.cc:507] Master 127.21.127.124:43885 requested a full tablet report, sending...
I20251210 21:03:38.184343 22173 ts_manager.cc:194] Registered new tserver with Master: fecaf19f8664480eaaffe1b3fe30508b (127.21.127.67:33635)
I20251210 21:03:38.187014 22479 heartbeater.cc:344] Connected to a master server at 127.21.127.126:39107
I20251210 21:03:38.187188 22479 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.187752 22479 heartbeater.cc:507] Master 127.21.127.126:39107 requested a full tablet report, sending...
I20251210 21:03:38.188961 22043 ts_manager.cc:194] Registered new tserver with Master: fecaf19f8664480eaaffe1b3fe30508b (127.21.127.67:33635)
I20251210 21:03:38.193238 22480 heartbeater.cc:344] Connected to a master server at 127.21.127.125:34635
I20251210 21:03:38.193401 22480 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:38.193926 22480 heartbeater.cc:507] Master 127.21.127.125:34635 requested a full tablet report, sending...
I20251210 21:03:38.194931 22108 ts_manager.cc:194] Registered new tserver with Master: fecaf19f8664480eaaffe1b3fe30508b (127.21.127.67:33635)
I20251210 21:03:38.195082 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.008174244s
I20251210 21:03:38.255110 22219 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:38.255501 22219 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:38.257288 22219 leader_election.cc:290] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c414d56a9c3e45a5bf6d9f4eeabc1b2c (127.21.127.126:39107), 90192a3a7f2f4d40a8667f08020c9b7e (127.21.127.124:43885)
I20251210 21:03:38.258090 22053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c609eed8924b3188cead6107717180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" is_pre_election: true
I20251210 21:03:38.258138 22183 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c609eed8924b3188cead6107717180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" is_pre_election: true
I20251210 21:03:38.258596 22053 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75c609eed8924b3188cead6107717180 in term 0.
I20251210 21:03:38.258603 22183 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75c609eed8924b3188cead6107717180 in term 0.
I20251210 21:03:38.259248 22092 leader_election.cc:304] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c609eed8924b3188cead6107717180, 90192a3a7f2f4d40a8667f08020c9b7e; no voters:
I20251210 21:03:38.259642 22219 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:38.259810 22219 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:38.259932 22219 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:38.262290 22219 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:38.262965 22219 leader_election.cc:290] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [CANDIDATE]: Term 1 election: Requested vote from peers c414d56a9c3e45a5bf6d9f4eeabc1b2c (127.21.127.126:39107), 90192a3a7f2f4d40a8667f08020c9b7e (127.21.127.124:43885)
I20251210 21:03:38.263489 22053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c609eed8924b3188cead6107717180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c"
I20251210 21:03:38.263535 22183 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75c609eed8924b3188cead6107717180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90192a3a7f2f4d40a8667f08020c9b7e"
I20251210 21:03:38.263680 22053 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:38.263736 22183 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:38.266027 22053 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75c609eed8924b3188cead6107717180 in term 1.
I20251210 21:03:38.266453 22092 leader_election.cc:304] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c609eed8924b3188cead6107717180, c414d56a9c3e45a5bf6d9f4eeabc1b2c; no voters:
I20251210 21:03:38.266781 22219 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:38.266884 22183 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75c609eed8924b3188cead6107717180 in term 1.
I20251210 21:03:38.275940 22219 raft_consensus.cc:697] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 1 LEADER]: Becoming Leader. State: Replica: 75c609eed8924b3188cead6107717180, State: Running, Role: LEADER
I20251210 21:03:38.276621 22219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } }
I20251210 21:03:38.280340 22485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c609eed8924b3188cead6107717180. Latest consensus state: current_term: 1 leader_uuid: "75c609eed8924b3188cead6107717180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:38.280673 22485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:38.281577 22487 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:03:38.283429 22487 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:03:38.291826 22053 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 1 FOLLOWER]: Refusing update from remote peer 75c609eed8924b3188cead6107717180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:38.291831 22183 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 1 FOLLOWER]: Refusing update from remote peer 75c609eed8924b3188cead6107717180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:38.292725 22485 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:38.293041 22219 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:38.300853 22220 sys_catalog.cc:455] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c609eed8924b3188cead6107717180. Latest consensus state: current_term: 1 leader_uuid: "75c609eed8924b3188cead6107717180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:38.301192 22220 sys_catalog.cc:458] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:38.302652 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c609eed8924b3188cead6107717180. Latest consensus state: current_term: 1 leader_uuid: "75c609eed8924b3188cead6107717180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:38.302920 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:38.304916 22490 mvcc.cc:204] Tried to move back new op lower bound from 7231080932512333824 to 7231080932466409472. Current Snapshot: MvccSnapshot[applied={T|T < 7231080932512333824}]
I20251210 21:03:38.314664 22485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75c609eed8924b3188cead6107717180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:38.315001 22485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:38.315601 22485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75c609eed8924b3188cead6107717180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:38.315841 22485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:38.316073 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75c609eed8924b3188cead6107717180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:38.316287 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:38.317020 22487 catalog_manager.cc:1357] Generated new cluster ID: ef97eb9b13814c1f881f255fc0aa005b
I20251210 21:03:38.317142 22487 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:03:38.318584 22220 sys_catalog.cc:455] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75c609eed8924b3188cead6107717180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c414d56a9c3e45a5bf6d9f4eeabc1b2c" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 39107 } } peers { permanent_uuid: "75c609eed8924b3188cead6107717180" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 34635 } } peers { permanent_uuid: "90192a3a7f2f4d40a8667f08020c9b7e" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 43885 } } }
I20251210 21:03:38.318789 22220 sys_catalog.cc:458] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:38.329162 22487 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:03:38.330513 22487 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:03:38.345412 22487 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180: Generated new TSK 0
I20251210 21:03:38.346588 22487 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:03:38.956975 22245 catalog_manager.cc:1269] Loaded cluster ID: ef97eb9b13814c1f881f255fc0aa005b
I20251210 21:03:38.957078 22245 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: loading cluster ID for follower catalog manager: success
I20251210 21:03:38.959231 22245 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: acquiring CA information for follower catalog manager: success
I20251210 21:03:38.959250 22252 catalog_manager.cc:1269] Loaded cluster ID: ef97eb9b13814c1f881f255fc0aa005b
I20251210 21:03:38.959379 22252 catalog_manager.cc:1562] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: loading cluster ID for follower catalog manager: success
I20251210 21:03:38.960841 22245 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251210 21:03:38.961822 22252 catalog_manager.cc:1584] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: acquiring CA information for follower catalog manager: success
I20251210 21:03:38.962746 22252 catalog_manager.cc:1612] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251210 21:03:39.045995 22108 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40110
I20251210 21:03:39.120424 22108 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40122
I20251210 21:03:39.197042 22108 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40138
I20251210 21:03:39.944562 22233 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:03:39.947111 22013 test_util.cc:276] Using random seed: 1557566544
I20251210 21:03:39.971127 22108 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40148:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251210 21:03:40.015313 22289 tablet_service.cc:1505] Processing CreateTablet for tablet 283d8cf796e74138b6e3f861c8549b2c (DEFAULT_TABLE table=test-workload [id=30e192686841408fa3750494074a7304]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:03:40.015235 22366 tablet_service.cc:1505] Processing CreateTablet for tablet 3de298ee87cc4e57ae24e0cd1fc83dde (DEFAULT_TABLE table=test-workload [id=30e192686841408fa3750494074a7304]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:03:40.015266 22443 tablet_service.cc:1505] Processing CreateTablet for tablet 35e120fa3a2a47178f98962e1fb8b45b (DEFAULT_TABLE table=test-workload [id=30e192686841408fa3750494074a7304]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:03:40.016115 22289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283d8cf796e74138b6e3f861c8549b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:40.015182 22288 tablet_service.cc:1505] Processing CreateTablet for tablet b45500009fdb44398d1f59b2f5283af8 (DEFAULT_TABLE table=test-workload [id=30e192686841408fa3750494074a7304]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:03:40.016506 22443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e120fa3a2a47178f98962e1fb8b45b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:40.016563 22288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b45500009fdb44398d1f59b2f5283af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:40.016120 22366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3de298ee87cc4e57ae24e0cd1fc83dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:40.022667 22526 tablet_bootstrap.cc:492] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b: Bootstrap starting.
I20251210 21:03:40.025072 22526 tablet_bootstrap.cc:654] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:40.027398 22526 tablet_bootstrap.cc:492] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b: No bootstrap required, opened a new log
I20251210 21:03:40.027611 22526 ts_tablet_manager.cc:1403] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20251210 21:03:40.028575 22526 raft_consensus.cc:359] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecaf19f8664480eaaffe1b3fe30508b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 33635 } }
I20251210 21:03:40.028788 22526 raft_consensus.cc:385] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:40.028898 22526 raft_consensus.cc:740] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecaf19f8664480eaaffe1b3fe30508b, State: Initialized, Role: FOLLOWER
I20251210 21:03:40.029158 22526 consensus_queue.cc:260] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecaf19f8664480eaaffe1b3fe30508b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 33635 } }
I20251210 21:03:40.029312 22526 raft_consensus.cc:399] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:40.029403 22526 raft_consensus.cc:493] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:40.029510 22526 raft_consensus.cc:3060] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:40.030745 22527 tablet_bootstrap.cc:492] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4: Bootstrap starting.
I20251210 21:03:40.031472 22526 raft_consensus.cc:515] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecaf19f8664480eaaffe1b3fe30508b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 33635 } }
I20251210 21:03:40.031731 22526 leader_election.cc:304] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fecaf19f8664480eaaffe1b3fe30508b; no voters:
I20251210 21:03:40.032416 22527 tablet_bootstrap.cc:654] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:40.032445 22530 raft_consensus.cc:2804] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:40.033428 22530 raft_consensus.cc:697] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 1 LEADER]: Becoming Leader. State: Replica: fecaf19f8664480eaaffe1b3fe30508b, State: Running, Role: LEADER
I20251210 21:03:40.033640 22526 leader_election.cc:290] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:40.033864 22530 consensus_queue.cc:237] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecaf19f8664480eaaffe1b3fe30508b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 33635 } }
I20251210 21:03:40.034662 22480 heartbeater.cc:499] Master 127.21.127.125:34635 was elected leader, sending a full tablet report...
I20251210 21:03:40.035974 22527 tablet_bootstrap.cc:492] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4: No bootstrap required, opened a new log
I20251210 21:03:40.036111 22527 ts_tablet_manager.cc:1403] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251210 21:03:40.036752 22527 raft_consensus.cc:359] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.036901 22527 raft_consensus.cc:385] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:40.037002 22527 raft_consensus.cc:740] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 424fa9035a49437da441bcc03f182de4, State: Initialized, Role: FOLLOWER
I20251210 21:03:40.037217 22527 consensus_queue.cc:260] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.037359 22526 ts_tablet_manager.cc:1434] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b: Time spent starting tablet: real 0.010s user 0.000s sys 0.006s
I20251210 21:03:40.037381 22527 raft_consensus.cc:399] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:40.037500 22527 raft_consensus.cc:493] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:40.037616 22527 raft_consensus.cc:3060] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:40.038885 22529 tablet_bootstrap.cc:492] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc: Bootstrap starting.
I20251210 21:03:40.041883 22527 raft_consensus.cc:515] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.042131 22527 leader_election.cc:304] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 424fa9035a49437da441bcc03f182de4; no voters:
I20251210 21:03:40.042287 22529 tablet_bootstrap.cc:654] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:40.045652 22108 catalog_manager.cc:5654] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b reported cstate change: term changed from 0 to 1, leader changed from <none> to fecaf19f8664480eaaffe1b3fe30508b (127.21.127.67). New cstate: current_term: 1 leader_uuid: "fecaf19f8664480eaaffe1b3fe30508b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecaf19f8664480eaaffe1b3fe30508b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 33635 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:40.047726 22326 heartbeater.cc:499] Master 127.21.127.125:34635 was elected leader, sending a full tablet report...
I20251210 21:03:40.053026 22536 raft_consensus.cc:2804] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:40.053531 22527 leader_election.cc:290] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:40.054462 22536 raft_consensus.cc:697] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 1 LEADER]: Becoming Leader. State: Replica: 424fa9035a49437da441bcc03f182de4, State: Running, Role: LEADER
I20251210 21:03:40.054924 22536 consensus_queue.cc:237] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.058439 22527 ts_tablet_manager.cc:1434] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4: Time spent starting tablet: real 0.022s user 0.000s sys 0.014s
I20251210 21:03:40.058771 22527 tablet_bootstrap.cc:492] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4: Bootstrap starting.
I20251210 21:03:40.059250 22529 tablet_bootstrap.cc:492] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc: No bootstrap required, opened a new log
I20251210 21:03:40.059417 22529 ts_tablet_manager.cc:1403] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.013s
I20251210 21:03:40.061136 22527 tablet_bootstrap.cc:654] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:40.061945 22108 catalog_manager.cc:5654] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 424fa9035a49437da441bcc03f182de4 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "424fa9035a49437da441bcc03f182de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:40.064807 22529 raft_consensus.cc:359] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05dda0e81b545aa9021e5d9795ab9cc" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33997 } }
I20251210 21:03:40.064989 22529 raft_consensus.cc:385] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:40.065110 22529 raft_consensus.cc:740] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05dda0e81b545aa9021e5d9795ab9cc, State: Initialized, Role: FOLLOWER
I20251210 21:03:40.065348 22529 consensus_queue.cc:260] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05dda0e81b545aa9021e5d9795ab9cc" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33997 } }
I20251210 21:03:40.065532 22529 raft_consensus.cc:399] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:40.065616 22529 raft_consensus.cc:493] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:40.065723 22529 raft_consensus.cc:3060] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:40.068753 22529 raft_consensus.cc:515] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05dda0e81b545aa9021e5d9795ab9cc" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33997 } }
I20251210 21:03:40.069047 22529 leader_election.cc:304] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e05dda0e81b545aa9021e5d9795ab9cc; no voters:
I20251210 21:03:40.076332 22527 tablet_bootstrap.cc:492] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4: No bootstrap required, opened a new log
I20251210 21:03:40.076387 22543 raft_consensus.cc:2804] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:40.076591 22527 ts_tablet_manager.cc:1403] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.000s
I20251210 21:03:40.077014 22543 raft_consensus.cc:697] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 1 LEADER]: Becoming Leader. State: Replica: e05dda0e81b545aa9021e5d9795ab9cc, State: Running, Role: LEADER
I20251210 21:03:40.077369 22403 heartbeater.cc:499] Master 127.21.127.125:34635 was elected leader, sending a full tablet report...
I20251210 21:03:40.077517 22543 consensus_queue.cc:237] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05dda0e81b545aa9021e5d9795ab9cc" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33997 } }
I20251210 21:03:40.077980 22527 raft_consensus.cc:359] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.079474 22527 raft_consensus.cc:385] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:40.079587 22527 raft_consensus.cc:740] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 424fa9035a49437da441bcc03f182de4, State: Initialized, Role: FOLLOWER
I20251210 21:03:40.079854 22527 consensus_queue.cc:260] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.080014 22529 leader_election.cc:290] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:40.082818 22529 ts_tablet_manager.cc:1434] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc: Time spent starting tablet: real 0.021s user 0.007s sys 0.003s
I20251210 21:03:40.083213 22527 raft_consensus.cc:399] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:40.083380 22527 raft_consensus.cc:493] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:40.083549 22527 raft_consensus.cc:3060] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:40.086189 22106 catalog_manager.cc:5654] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc reported cstate change: term changed from 0 to 1, leader changed from <none> to e05dda0e81b545aa9021e5d9795ab9cc (127.21.127.66). New cstate: current_term: 1 leader_uuid: "e05dda0e81b545aa9021e5d9795ab9cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05dda0e81b545aa9021e5d9795ab9cc" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33997 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:40.109375 22527 raft_consensus.cc:515] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.109634 22527 leader_election.cc:304] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 424fa9035a49437da441bcc03f182de4; no voters:
I20251210 21:03:40.109869 22527 leader_election.cc:290] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:40.109930 22536 raft_consensus.cc:2804] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:40.110169 22536 raft_consensus.cc:697] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 1 LEADER]: Becoming Leader. State: Replica: 424fa9035a49437da441bcc03f182de4, State: Running, Role: LEADER
I20251210 21:03:40.110386 22527 ts_tablet_manager.cc:1434] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20251210 21:03:40.110425 22536 consensus_queue.cc:237] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } }
I20251210 21:03:40.112646 22106 catalog_manager.cc:5654] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 424fa9035a49437da441bcc03f182de4 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "424fa9035a49437da441bcc03f182de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424fa9035a49437da441bcc03f182de4" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 42837 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:40.147233 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:03:40.157383 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:40.157876 22013 tablet_replica.cc:333] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4: stopping tablet replica
I20251210 21:03:40.158268 22013 raft_consensus.cc:2243] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:40.158540 22013 raft_consensus.cc:2272] T 283d8cf796e74138b6e3f861c8549b2c P 424fa9035a49437da441bcc03f182de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:40.160202 22013 tablet_replica.cc:333] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4: stopping tablet replica
I20251210 21:03:40.160372 22013 raft_consensus.cc:2243] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:40.160537 22013 raft_consensus.cc:2272] T b45500009fdb44398d1f59b2f5283af8 P 424fa9035a49437da441bcc03f182de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:40.176535 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:03:40.181891 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:03:40.190848 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:40.191231 22013 tablet_replica.cc:333] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc: stopping tablet replica
I20251210 21:03:40.191470 22013 raft_consensus.cc:2243] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:40.191674 22013 raft_consensus.cc:2272] T 3de298ee87cc4e57ae24e0cd1fc83dde P e05dda0e81b545aa9021e5d9795ab9cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:40.195875 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:03:40.199035 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:03:40.207254 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:40.207679 22013 tablet_replica.cc:333] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b: stopping tablet replica
I20251210 21:03:40.207865 22013 raft_consensus.cc:2243] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:40.208057 22013 raft_consensus.cc:2272] T 35e120fa3a2a47178f98962e1fb8b45b P fecaf19f8664480eaaffe1b3fe30508b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:40.222397 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:03:40.225576 22013 master.cc:561] Master@127.21.127.126:39107 shutting down...
I20251210 21:03:40.234303 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:40.234593 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:40.234755 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P c414d56a9c3e45a5bf6d9f4eeabc1b2c: stopping tablet replica
I20251210 21:03:40.249347 22013 master.cc:583] Master@127.21.127.126:39107 shutdown complete.
I20251210 21:03:40.254253 22013 master.cc:561] Master@127.21.127.125:34635 shutting down...
I20251210 21:03:40.261435 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:40.261940 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:40.262079 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 75c609eed8924b3188cead6107717180: stopping tablet replica
I20251210 21:03:40.277294 22013 master.cc:583] Master@127.21.127.125:34635 shutdown complete.
I20251210 21:03:40.281666 22013 master.cc:561] Master@127.21.127.124:43885 shutting down...
I20251210 21:03:40.289009 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:40.289254 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:40.289387 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 90192a3a7f2f4d40a8667f08020c9b7e: stopping tablet replica
I20251210 21:03:40.303402 22013 master.cc:583] Master@127.21.127.124:43885 shutdown complete.
[ OK ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing (2608 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn
I20251210 21:03:40.316076 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:43953
I20251210 21:03:40.316617 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:40.318890 22548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.318917 22549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.319485 22551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:40.319824 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:40.320083 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:40.320159 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:40.320210 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400620320194 us; error 0 us; skew 500 ppm
I20251210 21:03:40.320379 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:40.321467 22013 webserver.cc:492] Webserver started at http://127.21.127.126:38731/ using document root <none> and password file <none>
I20251210 21:03:40.321718 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:40.321949 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:40.322145 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:40.322842 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "5f99f3ef87f64a7faf4d7825a1b1d877"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.325409 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:03:40.327170 22556 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.327572 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:40.327678 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "5f99f3ef87f64a7faf4d7825a1b1d877"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.327783 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:40.354120 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:40.354699 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:40.368999 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:43953
I20251210 21:03:40.369014 22607 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:43953 every 8 connection(s)
I20251210 21:03:40.370416 22608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:40.374409 22608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877: Bootstrap starting.
I20251210 21:03:40.375860 22608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:40.377483 22608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877: No bootstrap required, opened a new log
I20251210 21:03:40.378141 22608 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" member_type: VOTER }
I20251210 21:03:40.378284 22608 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:40.378350 22608 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f99f3ef87f64a7faf4d7825a1b1d877, State: Initialized, Role: FOLLOWER
I20251210 21:03:40.378558 22608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" member_type: VOTER }
I20251210 21:03:40.378690 22608 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:40.378762 22608 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:40.378873 22608 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:40.380684 22608 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" member_type: VOTER }
I20251210 21:03:40.380900 22608 leader_election.cc:304] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f99f3ef87f64a7faf4d7825a1b1d877; no voters:
I20251210 21:03:40.381369 22608 leader_election.cc:290] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:40.381518 22611 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:40.382107 22611 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 1 LEADER]: Becoming Leader. State: Replica: 5f99f3ef87f64a7faf4d7825a1b1d877, State: Running, Role: LEADER
I20251210 21:03:40.382242 22608 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:40.382382 22611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" member_type: VOTER }
I20251210 21:03:40.383311 22612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" member_type: VOTER } }
I20251210 21:03:40.383369 22613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f99f3ef87f64a7faf4d7825a1b1d877. Latest consensus state: current_term: 1 leader_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f99f3ef87f64a7faf4d7825a1b1d877" member_type: VOTER } }
I20251210 21:03:40.383587 22613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:40.383597 22612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:40.387224 22616 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:03:40.387517 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:03:40.388779 22616 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:03:40.391901 22616 catalog_manager.cc:1357] Generated new cluster ID: b32c6a4dffea4bc0b8b31cf6e9393354
I20251210 21:03:40.391978 22616 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:03:40.422149 22616 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:03:40.422930 22616 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:03:40.429080 22616 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877: Generated new TSK 0
I20251210 21:03:40.429327 22616 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:03:40.452839 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:40.455943 22630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.456072 22629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.457325 22632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:40.457585 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:40.457892 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:40.457978 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:40.458056 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400620458018 us; error 0 us; skew 500 ppm
I20251210 21:03:40.458211 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:40.459486 22013 webserver.cc:492] Webserver started at http://127.21.127.65:40415/ using document root <none> and password file <none>
I20251210 21:03:40.459718 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:40.459874 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:40.460110 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:40.460841 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "32c994ccef9b4615b5b6df3e8200bde6"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.463392 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20251210 21:03:40.465122 22637 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.465484 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251210 21:03:40.465601 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "32c994ccef9b4615b5b6df3e8200bde6"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.465730 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:40.489640 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:40.490283 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:40.490869 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:40.491788 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:40.491869 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.491979 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:40.492031 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.505635 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:37235
I20251210 21:03:40.505656 22699 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:37235 every 8 connection(s)
I20251210 21:03:40.507238 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:40.510485 22704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.511005 22705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.513576 22707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:40.515887 22700 heartbeater.cc:344] Connected to a master server at 127.21.127.126:43953
I20251210 21:03:40.516021 22700 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:40.516450 22700 heartbeater.cc:507] Master 127.21.127.126:43953 requested a full tablet report, sending...
I20251210 21:03:40.516727 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:40.517015 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:40.517107 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:40.517170 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400620517148 us; error 0 us; skew 500 ppm
I20251210 21:03:40.517354 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:40.517508 22573 ts_manager.cc:194] Registered new tserver with Master: 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235)
I20251210 21:03:40.518646 22013 webserver.cc:492] Webserver started at http://127.21.127.66:39797/ using document root <none> and password file <none>
I20251210 21:03:40.518894 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:40.519003 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:40.518976 22573 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39220
I20251210 21:03:40.519183 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:40.519995 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "7aa6710b99204722af28432ffe9920f0"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.522377 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:03:40.523943 22712 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.524276 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:03:40.524394 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "7aa6710b99204722af28432ffe9920f0"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.524511 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:40.544487 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:40.545112 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:40.545689 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:40.546533 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:40.546620 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.546715 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:40.546789 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.559593 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:44985
I20251210 21:03:40.559639 22774 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:44985 every 8 connection(s)
I20251210 21:03:40.561228 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:40.564774 22778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.565220 22779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:40.565450 22781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:40.566111 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:40.566402 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:40.566511 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:40.566614 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400620566567 us; error 0 us; skew 500 ppm
I20251210 21:03:40.566802 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:40.568118 22013 webserver.cc:492] Webserver started at http://127.21.127.67:34317/ using document root <none> and password file <none>
I20251210 21:03:40.568365 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:40.568481 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:40.568629 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:40.568706 22775 heartbeater.cc:344] Connected to a master server at 127.21.127.126:43953
I20251210 21:03:40.568868 22775 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:40.569250 22775 heartbeater.cc:507] Master 127.21.127.126:43953 requested a full tablet report, sending...
I20251210 21:03:40.569428 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "9e5905594330410887c8f13761b31e2f"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.570181 22573 ts_manager.cc:194] Registered new tserver with Master: 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985)
I20251210 21:03:40.571147 22573 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39224
I20251210 21:03:40.572492 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251210 21:03:40.574067 22786 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.574450 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:03:40.574555 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "9e5905594330410887c8f13761b31e2f"
format_stamp: "Formatted at 2025-12-10 21:03:40 on dist-test-slave-9pq8"
I20251210 21:03:40.574685 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:40.586076 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:40.586691 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:40.587231 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:40.588162 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:40.588227 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.588338 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:40.588382 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:40.601543 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:46615
I20251210 21:03:40.601573 22848 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:46615 every 8 connection(s)
I20251210 21:03:40.608341 22849 heartbeater.cc:344] Connected to a master server at 127.21.127.126:43953
I20251210 21:03:40.608492 22849 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:40.608893 22849 heartbeater.cc:507] Master 127.21.127.126:43953 requested a full tablet report, sending...
I20251210 21:03:40.609733 22573 ts_manager.cc:194] Registered new tserver with Master: 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:40.610425 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00798868s
I20251210 21:03:40.610510 22573 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39238
I20251210 21:03:41.520450 22700 heartbeater.cc:499] Master 127.21.127.126:43953 was elected leader, sending a full tablet report...
I20251210 21:03:41.572453 22775 heartbeater.cc:499] Master 127.21.127.126:43953 was elected leader, sending a full tablet report...
I20251210 21:03:41.611796 22849 heartbeater.cc:499] Master 127.21.127.126:43953 was elected leader, sending a full tablet report...
I20251210 21:03:41.649963 22013 test_util.cc:276] Using random seed: 1559269395
I20251210 21:03:41.663116 22573 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39246:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o""\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:03:41.663949 22573 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:03:41.693943 22740 tablet_service.cc:1505] Processing CreateTablet for tablet 01df683a26744048a50073eafe485b5a (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251210 21:03:41.694314 22739 tablet_service.cc:1505] Processing CreateTablet for tablet 666c53fa653843e9b3f6ba05a3bb7de7 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251210 21:03:41.694598 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01df683a26744048a50073eafe485b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.694806 22739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666c53fa653843e9b3f6ba05a3bb7de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.694684 22738 tablet_service.cc:1505] Processing CreateTablet for tablet cfb1b71eb54349289ce17d8812199f90 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251210 21:03:41.695003 22661 tablet_service.cc:1505] Processing CreateTablet for tablet 3b25e7bfefbf4c7c98791d0cf104a984 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251210 21:03:41.695210 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb1b71eb54349289ce17d8812199f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.695899 22658 tablet_service.cc:1505] Processing CreateTablet for tablet 044baf974d174b988cdf96de51b766cf (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251210 21:03:41.696272 22661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b25e7bfefbf4c7c98791d0cf104a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.698452 22659 tablet_service.cc:1505] Processing CreateTablet for tablet bacefe14d5e64db592b052b9f269c244 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251210 21:03:41.698915 22659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bacefe14d5e64db592b052b9f269c244. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.699916 22736 tablet_service.cc:1505] Processing CreateTablet for tablet 3b25e7bfefbf4c7c98791d0cf104a984 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251210 21:03:41.699954 22662 tablet_service.cc:1505] Processing CreateTablet for tablet 55294cadb3be456fb164afeff111746e (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251210 21:03:41.700400 22736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b25e7bfefbf4c7c98791d0cf104a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.700474 22662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55294cadb3be456fb164afeff111746e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.702621 22658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044baf974d174b988cdf96de51b766cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.703948 22734 tablet_service.cc:1505] Processing CreateTablet for tablet bacefe14d5e64db592b052b9f269c244 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251210 21:03:41.704263 22733 tablet_service.cc:1505] Processing CreateTablet for tablet 044baf974d174b988cdf96de51b766cf (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251210 21:03:41.704425 22734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bacefe14d5e64db592b052b9f269c244. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.704738 22733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044baf974d174b988cdf96de51b766cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.705044 22664 tablet_service.cc:1505] Processing CreateTablet for tablet 666c53fa653843e9b3f6ba05a3bb7de7 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251210 21:03:41.705467 22664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666c53fa653843e9b3f6ba05a3bb7de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.707952 22735 tablet_service.cc:1505] Processing CreateTablet for tablet 11941900d8c24629b869a9bd16ecb96b (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251210 21:03:41.709101 22663 tablet_service.cc:1505] Processing CreateTablet for tablet cfb1b71eb54349289ce17d8812199f90 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251210 21:03:41.710330 22665 tablet_service.cc:1505] Processing CreateTablet for tablet 01df683a26744048a50073eafe485b5a (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251210 21:03:41.711648 22663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb1b71eb54349289ce17d8812199f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.719318 22737 tablet_service.cc:1505] Processing CreateTablet for tablet 55294cadb3be456fb164afeff111746e (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251210 21:03:41.719884 22660 tablet_service.cc:1505] Processing CreateTablet for tablet 11941900d8c24629b869a9bd16ecb96b (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251210 21:03:41.720402 22660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11941900d8c24629b869a9bd16ecb96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.721158 22814 tablet_service.cc:1505] Processing CreateTablet for tablet 01df683a26744048a50073eafe485b5a (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251210 21:03:41.721274 22813 tablet_service.cc:1505] Processing CreateTablet for tablet 666c53fa653843e9b3f6ba05a3bb7de7 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251210 21:03:41.721582 22814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01df683a26744048a50073eafe485b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.721830 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666c53fa653843e9b3f6ba05a3bb7de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.722077 22665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01df683a26744048a50073eafe485b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.722913 22812 tablet_service.cc:1505] Processing CreateTablet for tablet cfb1b71eb54349289ce17d8812199f90 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251210 21:03:41.723378 22735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11941900d8c24629b869a9bd16ecb96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.723881 22812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb1b71eb54349289ce17d8812199f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.724651 22811 tablet_service.cc:1505] Processing CreateTablet for tablet 55294cadb3be456fb164afeff111746e (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251210 21:03:41.725170 22811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55294cadb3be456fb164afeff111746e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.726486 22810 tablet_service.cc:1505] Processing CreateTablet for tablet 3b25e7bfefbf4c7c98791d0cf104a984 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251210 21:03:41.730057 22737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55294cadb3be456fb164afeff111746e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.731395 22809 tablet_service.cc:1505] Processing CreateTablet for tablet 11941900d8c24629b869a9bd16ecb96b (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251210 21:03:41.734485 22807 tablet_service.cc:1505] Processing CreateTablet for tablet 044baf974d174b988cdf96de51b766cf (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251210 21:03:41.733692 22808 tablet_service.cc:1505] Processing CreateTablet for tablet bacefe14d5e64db592b052b9f269c244 (DEFAULT_TABLE table=test-workload [id=63230cea32054744a182d2a35fb94ab1]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251210 21:03:41.736387 22807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044baf974d174b988cdf96de51b766cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.736394 22869 tablet_bootstrap.cc:492] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.738236 22870 tablet_bootstrap.cc:492] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.740314 22810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b25e7bfefbf4c7c98791d0cf104a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.743835 22869 tablet_bootstrap.cc:654] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.744968 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11941900d8c24629b869a9bd16ecb96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.745790 22808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bacefe14d5e64db592b052b9f269c244. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:41.781677 22869 tablet_bootstrap.cc:492] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.781911 22869 ts_tablet_manager.cc:1403] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.012s
I20251210 21:03:41.787878 22869 raft_consensus.cc:359] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.789769 22870 tablet_bootstrap.cc:654] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.789842 22872 tablet_bootstrap.cc:492] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.789803 22869 raft_consensus.cc:385] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.791594 22869 raft_consensus.cc:740] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.791934 22869 consensus_queue.cc:260] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.792845 22869 ts_tablet_manager.cc:1434] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.011s user 0.002s sys 0.002s
I20251210 21:03:41.793144 22869 tablet_bootstrap.cc:492] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.793447 22872 tablet_bootstrap.cc:654] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.794895 22869 tablet_bootstrap.cc:654] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.799423 22872 tablet_bootstrap.cc:492] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.799502 22870 tablet_bootstrap.cc:492] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.799696 22872 ts_tablet_manager.cc:1403] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20251210 21:03:41.799739 22870 ts_tablet_manager.cc:1403] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.062s user 0.004s sys 0.008s
I20251210 21:03:41.800437 22872 raft_consensus.cc:359] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.800624 22872 raft_consensus.cc:385] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.800623 22870 raft_consensus.cc:359] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.800827 22872 raft_consensus.cc:740] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.800872 22870 raft_consensus.cc:385] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.800983 22870 raft_consensus.cc:740] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.801092 22872 consensus_queue.cc:260] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.801208 22870 consensus_queue.cc:260] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.801949 22872 ts_tablet_manager.cc:1434] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:03:41.802282 22872 tablet_bootstrap.cc:492] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.803157 22869 tablet_bootstrap.cc:492] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.803346 22869 ts_tablet_manager.cc:1403] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20251210 21:03:41.804169 22872 tablet_bootstrap.cc:654] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.804239 22869 raft_consensus.cc:359] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.804471 22869 raft_consensus.cc:385] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.804549 22869 raft_consensus.cc:740] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.804798 22869 consensus_queue.cc:260] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.807688 22869 ts_tablet_manager.cc:1434] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:03:41.808847 22869 tablet_bootstrap.cc:492] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.810954 22869 tablet_bootstrap.cc:654] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.810959 22870 ts_tablet_manager.cc:1434] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.011s user 0.000s sys 0.011s
I20251210 21:03:41.811297 22870 tablet_bootstrap.cc:492] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.813470 22870 tablet_bootstrap.cc:654] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.815409 22872 tablet_bootstrap.cc:492] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.815627 22872 ts_tablet_manager.cc:1403] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.002s
I20251210 21:03:41.816440 22872 raft_consensus.cc:359] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.816617 22872 raft_consensus.cc:385] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.816699 22872 raft_consensus.cc:740] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.816812 22869 tablet_bootstrap.cc:492] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.816929 22872 consensus_queue.cc:260] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.817055 22869 ts_tablet_manager.cc:1403] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.002s
I20251210 21:03:41.817409 22872 ts_tablet_manager.cc:1434] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:03:41.817703 22872 tablet_bootstrap.cc:492] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.817823 22869 raft_consensus.cc:359] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.818029 22869 raft_consensus.cc:385] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.818121 22869 raft_consensus.cc:740] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.818416 22869 consensus_queue.cc:260] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.819141 22869 ts_tablet_manager.cc:1434] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:03:41.819885 22869 tablet_bootstrap.cc:492] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.821344 22872 tablet_bootstrap.cc:654] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.822021 22870 tablet_bootstrap.cc:492] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.822202 22870 ts_tablet_manager.cc:1403] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20251210 21:03:41.822291 22869 tablet_bootstrap.cc:654] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.823129 22870 raft_consensus.cc:359] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.823613 22870 raft_consensus.cc:385] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.823721 22870 raft_consensus.cc:740] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.823972 22870 consensus_queue.cc:260] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.824455 22869 tablet_bootstrap.cc:492] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.824532 22870 ts_tablet_manager.cc:1434] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:03:41.824623 22869 ts_tablet_manager.cc:1403] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:03:41.824888 22872 tablet_bootstrap.cc:492] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.824919 22870 tablet_bootstrap.cc:492] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.825035 22872 ts_tablet_manager.cc:1403] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20251210 21:03:41.825455 22869 raft_consensus.cc:359] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.825650 22869 raft_consensus.cc:385] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.825764 22869 raft_consensus.cc:740] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.825778 22872 raft_consensus.cc:359] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.826007 22872 raft_consensus.cc:385] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.826104 22872 raft_consensus.cc:740] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.826033 22869 consensus_queue.cc:260] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.826397 22872 consensus_queue.cc:260] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.826546 22869 ts_tablet_manager.cc:1434] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:03:41.826820 22870 tablet_bootstrap.cc:654] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.827023 22872 ts_tablet_manager.cc:1434] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:03:41.827286 22872 tablet_bootstrap.cc:492] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.827323 22869 tablet_bootstrap.cc:492] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.829231 22869 tablet_bootstrap.cc:654] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.829249 22872 tablet_bootstrap.cc:654] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.834414 22870 tablet_bootstrap.cc:492] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.834597 22870 ts_tablet_manager.cc:1403] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20251210 21:03:41.834887 22872 tablet_bootstrap.cc:492] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.835044 22872 ts_tablet_manager.cc:1403] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20251210 21:03:41.835474 22870 raft_consensus.cc:359] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.835690 22870 raft_consensus.cc:385] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.835810 22870 raft_consensus.cc:740] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.835741 22872 raft_consensus.cc:359] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.835999 22872 raft_consensus.cc:385] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.836090 22872 raft_consensus.cc:740] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.836117 22870 consensus_queue.cc:260] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.836304 22872 consensus_queue.cc:260] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.836659 22870 ts_tablet_manager.cc:1434] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:03:41.836778 22872 ts_tablet_manager.cc:1434] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:03:41.836966 22870 tablet_bootstrap.cc:492] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.837126 22869 tablet_bootstrap.cc:492] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.837381 22869 ts_tablet_manager.cc:1403] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20251210 21:03:41.838189 22869 raft_consensus.cc:359] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.838423 22869 raft_consensus.cc:385] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.838577 22869 raft_consensus.cc:740] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.838723 22870 tablet_bootstrap.cc:654] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.837241 22872 tablet_bootstrap.cc:492] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.840540 22872 tablet_bootstrap.cc:654] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.840848 22870 tablet_bootstrap.cc:492] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.841029 22870 ts_tablet_manager.cc:1403] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:03:41.841534 22869 consensus_queue.cc:260] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.841753 22870 raft_consensus.cc:359] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.842044 22874 raft_consensus.cc:493] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:41.842152 22869 ts_tablet_manager.cc:1434] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251210 21:03:41.842198 22874 raft_consensus.cc:515] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.842506 22869 tablet_bootstrap.cc:492] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.843322 22874 leader_election.cc:290] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:41.843549 22872 tablet_bootstrap.cc:492] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.843703 22872 ts_tablet_manager.cc:1403] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251210 21:03:41.844367 22869 tablet_bootstrap.cc:654] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.844503 22872 raft_consensus.cc:359] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.844699 22872 raft_consensus.cc:385] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.844808 22872 raft_consensus.cc:740] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.845046 22872 consensus_queue.cc:260] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.845538 22872 ts_tablet_manager.cc:1434] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:03:41.845808 22872 tablet_bootstrap.cc:492] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.847560 22872 tablet_bootstrap.cc:654] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.848989 22870 raft_consensus.cc:385] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.849113 22870 raft_consensus.cc:740] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.849706 22870 consensus_queue.cc:260] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.850734 22870 ts_tablet_manager.cc:1434] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20251210 21:03:41.851068 22870 tablet_bootstrap.cc:492] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.851946 22874 raft_consensus.cc:493] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:41.852123 22874 raft_consensus.cc:515] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.852757 22874 leader_election.cc:290] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:41.855481 22870 tablet_bootstrap.cc:654] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.861102 22870 tablet_bootstrap.cc:492] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.861284 22870 ts_tablet_manager.cc:1403] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20251210 21:03:41.862164 22870 raft_consensus.cc:359] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.862401 22870 raft_consensus.cc:385] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.862501 22870 raft_consensus.cc:740] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.864759 22870 consensus_queue.cc:260] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.866957 22870 ts_tablet_manager.cc:1434] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20251210 21:03:41.867306 22870 tablet_bootstrap.cc:492] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.869374 22870 tablet_bootstrap.cc:654] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.873512 22869 tablet_bootstrap.cc:492] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.873710 22869 ts_tablet_manager.cc:1403] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.015s
I20251210 21:03:41.874562 22869 raft_consensus.cc:359] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.874789 22869 raft_consensus.cc:385] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.874884 22869 raft_consensus.cc:740] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.875190 22869 consensus_queue.cc:260] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.875747 22869 ts_tablet_manager.cc:1434] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20251210 21:03:41.876082 22869 tablet_bootstrap.cc:492] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.877964 22869 tablet_bootstrap.cc:654] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.880354 22870 tablet_bootstrap.cc:492] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.880543 22870 ts_tablet_manager.cc:1403] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20251210 21:03:41.881394 22870 raft_consensus.cc:359] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.881626 22870 raft_consensus.cc:385] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.881703 22870 raft_consensus.cc:740] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.882035 22870 consensus_queue.cc:260] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.882714 22870 ts_tablet_manager.cc:1434] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20251210 21:03:41.883049 22870 tablet_bootstrap.cc:492] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.885103 22870 tablet_bootstrap.cc:654] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.887326 22869 tablet_bootstrap.cc:492] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.887521 22869 ts_tablet_manager.cc:1403] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.008s
I20251210 21:03:41.888444 22869 raft_consensus.cc:359] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.888661 22869 raft_consensus.cc:385] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.888779 22869 raft_consensus.cc:740] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.889041 22869 consensus_queue.cc:260] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.889546 22869 ts_tablet_manager.cc:1434] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:03:41.889837 22869 tablet_bootstrap.cc:492] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6: Bootstrap starting.
I20251210 21:03:41.890784 22824 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "044baf974d174b988cdf96de51b766cf" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f" is_pre_election: true
I20251210 21:03:41.890843 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f" is_pre_election: true
I20251210 21:03:41.891769 22869 tablet_bootstrap.cc:654] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6: Neither blocks nor log segments found. Creating new log.
W20251210 21:03:41.892397 22640 leader_election.cc:343] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:03:41.893393 22870 tablet_bootstrap.cc:492] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.893554 22870 ts_tablet_manager.cc:1403] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20251210 21:03:41.894387 22870 raft_consensus.cc:359] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.894567 22870 raft_consensus.cc:385] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.894655 22870 raft_consensus.cc:740] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.894922 22870 consensus_queue.cc:260] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
W20251210 21:03:41.895413 22640 leader_election.cc:343] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:03:41.897797 22870 ts_tablet_manager.cc:1434] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251210 21:03:41.900246 22750 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "044baf974d174b988cdf96de51b766cf" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0" is_pre_election: true
I20251210 21:03:41.900600 22750 raft_consensus.cc:2468] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 0.
I20251210 21:03:41.901788 22870 tablet_bootstrap.cc:492] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0: Bootstrap starting.
I20251210 21:03:41.902256 22641 leader_election.cc:304] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0; no voters: 9e5905594330410887c8f13761b31e2f
I20251210 21:03:41.901507 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0" is_pre_election: true
I20251210 21:03:41.902513 22749 raft_consensus.cc:2468] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 0.
I20251210 21:03:41.902583 22874 raft_consensus.cc:2804] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:41.902694 22874 raft_consensus.cc:493] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:41.902796 22874 raft_consensus.cc:3060] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.903545 22869 tablet_bootstrap.cc:492] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6: No bootstrap required, opened a new log
I20251210 21:03:41.903728 22872 tablet_bootstrap.cc:492] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.903728 22869 ts_tablet_manager.cc:1403] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20251210 21:03:41.903910 22872 ts_tablet_manager.cc:1403] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.058s user 0.023s sys 0.034s
I20251210 21:03:41.904560 22869 raft_consensus.cc:359] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.904619 22872 raft_consensus.cc:359] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.904765 22869 raft_consensus.cc:385] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.904856 22869 raft_consensus.cc:740] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.904853 22872 raft_consensus.cc:385] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.905018 22872 raft_consensus.cc:740] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.905143 22869 consensus_queue.cc:260] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.905251 22872 consensus_queue.cc:260] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.905685 22872 ts_tablet_manager.cc:1434] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:03:41.905781 22874 raft_consensus.cc:515] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.906029 22872 tablet_bootstrap.cc:492] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.906786 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "044baf974d174b988cdf96de51b766cf" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0"
I20251210 21:03:41.906973 22749 raft_consensus.cc:3060] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.907608 22869 ts_tablet_manager.cc:1434] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251210 21:03:41.907975 22872 tablet_bootstrap.cc:654] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.909781 22749 raft_consensus.cc:2468] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:41.910043 22872 tablet_bootstrap.cc:492] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.910070 22641 leader_election.cc:304] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0; no voters: 9e5905594330410887c8f13761b31e2f
I20251210 21:03:41.910194 22872 ts_tablet_manager.cc:1403] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:03:41.910496 22641 leader_election.cc:304] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0; no voters:
I20251210 21:03:41.910880 22889 raft_consensus.cc:2804] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:41.910975 22889 raft_consensus.cc:493] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:41.911077 22889 raft_consensus.cc:3060] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.912153 22872 raft_consensus.cc:359] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.912314 22872 raft_consensus.cc:385] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.912388 22872 raft_consensus.cc:740] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.912559 22872 consensus_queue.cc:260] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.912997 22872 ts_tablet_manager.cc:1434] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251210 21:03:41.913270 22872 tablet_bootstrap.cc:492] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f: Bootstrap starting.
I20251210 21:03:41.913395 22889 raft_consensus.cc:515] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.913926 22889 leader_election.cc:290] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Requested vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:41.914108 22889 raft_consensus.cc:2804] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:41.914256 22889 raft_consensus.cc:697] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Becoming Leader. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Running, Role: LEADER
I20251210 21:03:41.914386 22824 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f"
I20251210 21:03:41.914562 22824 raft_consensus.cc:3060] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.914520 22889 consensus_queue.cc:237] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.915149 22872 tablet_bootstrap.cc:654] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.915341 22874 leader_election.cc:290] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Requested vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:41.915848 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "044baf974d174b988cdf96de51b766cf" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f"
I20251210 21:03:41.916041 22823 raft_consensus.cc:3060] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.917929 22824 raft_consensus.cc:2468] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:41.919965 22640 leader_election.cc:304] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:41.920745 22874 raft_consensus.cc:2804] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:41.920962 22874 raft_consensus.cc:697] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Becoming Leader. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Running, Role: LEADER
I20251210 21:03:41.921294 22874 consensus_queue.cc:237] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.921504 22823 raft_consensus.cc:2468] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:41.925297 22571 catalog_manager.cc:5654] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "32c994ccef9b4615b5b6df3e8200bde6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:41.926452 22870 tablet_bootstrap.cc:654] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:41.927587 22872 tablet_bootstrap.cc:492] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f: No bootstrap required, opened a new log
I20251210 21:03:41.927666 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0"
I20251210 21:03:41.927798 22872 ts_tablet_manager.cc:1403] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.013s
I20251210 21:03:41.927882 22749 raft_consensus.cc:3060] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.928654 22872 raft_consensus.cc:359] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.928901 22872 raft_consensus.cc:385] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.928989 22872 raft_consensus.cc:740] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.929303 22872 consensus_queue.cc:260] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.929852 22872 ts_tablet_manager.cc:1434] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:03:41.932637 22749 raft_consensus.cc:2468] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:41.934587 22572 catalog_manager.cc:5654] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "32c994ccef9b4615b5b6df3e8200bde6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:41.935683 22870 tablet_bootstrap.cc:492] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0: No bootstrap required, opened a new log
I20251210 21:03:41.935863 22870 ts_tablet_manager.cc:1403] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.005s
I20251210 21:03:41.936533 22870 raft_consensus.cc:359] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.936694 22870 raft_consensus.cc:385] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:41.936774 22870 raft_consensus.cc:740] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Initialized, Role: FOLLOWER
I20251210 21:03:41.937107 22870 consensus_queue.cc:260] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.937630 22870 ts_tablet_manager.cc:1434] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:03:41.972335 22877 raft_consensus.cc:493] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:41.972543 22877 raft_consensus.cc:515] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.973547 22877 leader_election.cc:290] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235), 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985)
I20251210 21:03:41.977447 22877 raft_consensus.cc:493] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:41.977633 22877 raft_consensus.cc:515] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.978338 22877 leader_election.cc:290] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235)
I20251210 21:03:41.981415 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfb1b71eb54349289ce17d8812199f90" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0" is_pre_election: true
I20251210 21:03:41.981490 22750 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01df683a26744048a50073eafe485b5a" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0" is_pre_election: true
I20251210 21:03:41.981336 22675 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfb1b71eb54349289ce17d8812199f90" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6" is_pre_election: true
I20251210 21:03:41.981535 22674 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01df683a26744048a50073eafe485b5a" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6" is_pre_election: true
I20251210 21:03:41.981691 22749 raft_consensus.cc:2468] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 0.
I20251210 21:03:41.981696 22750 raft_consensus.cc:2468] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 0.
I20251210 21:03:41.981774 22675 raft_consensus.cc:2468] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 0.
I20251210 21:03:41.981806 22674 raft_consensus.cc:2468] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 0.
I20251210 21:03:41.982203 22790 leader_election.cc:304] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa6710b99204722af28432ffe9920f0, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:41.982470 22877 raft_consensus.cc:2804] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:41.982501 22787 leader_election.cc:304] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:41.982563 22877 raft_consensus.cc:493] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:41.982671 22877 raft_consensus.cc:3060] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.983170 22907 raft_consensus.cc:2804] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:41.983273 22907 raft_consensus.cc:493] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:41.983402 22907 raft_consensus.cc:3060] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.985754 22877 raft_consensus.cc:515] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.986279 22907 raft_consensus.cc:515] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.986444 22877 leader_election.cc:290] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 election: Requested vote from peers 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235), 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985)
I20251210 21:03:41.986917 22674 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfb1b71eb54349289ce17d8812199f90" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6"
I20251210 21:03:41.987023 22750 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfb1b71eb54349289ce17d8812199f90" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0"
I20251210 21:03:41.987126 22674 raft_consensus.cc:3060] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.987181 22750 raft_consensus.cc:3060] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.987301 22675 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01df683a26744048a50073eafe485b5a" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6"
I20251210 21:03:41.987509 22675 raft_consensus.cc:3060] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.987617 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01df683a26744048a50073eafe485b5a" candidate_uuid: "9e5905594330410887c8f13761b31e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0"
I20251210 21:03:41.987771 22749 raft_consensus.cc:3060] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:41.990362 22674 raft_consensus.cc:2468] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 1.
I20251210 21:03:41.990389 22750 raft_consensus.cc:2468] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 1.
I20251210 21:03:41.990695 22907 leader_election.cc:290] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 election: Requested vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235)
I20251210 21:03:41.990736 22749 raft_consensus.cc:2468] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 1.
I20251210 21:03:41.990820 22675 raft_consensus.cc:2468] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5905594330410887c8f13761b31e2f in term 1.
I20251210 21:03:41.990957 22790 leader_election.cc:304] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa6710b99204722af28432ffe9920f0, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:41.991200 22907 raft_consensus.cc:2804] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:41.991263 22790 leader_election.cc:304] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa6710b99204722af28432ffe9920f0, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:41.991365 22907 raft_consensus.cc:697] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 1 LEADER]: Becoming Leader. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Running, Role: LEADER
I20251210 21:03:41.991544 22877 raft_consensus.cc:2804] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:41.991698 22877 raft_consensus.cc:697] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 1 LEADER]: Becoming Leader. State: Replica: 9e5905594330410887c8f13761b31e2f, State: Running, Role: LEADER
I20251210 21:03:41.991674 22907 consensus_queue.cc:237] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:41.991998 22877 consensus_queue.cc:237] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } }
I20251210 21:03:41.994884 22572 catalog_manager.cc:5654] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e5905594330410887c8f13761b31e2f (127.21.127.67). New cstate: current_term: 1 leader_uuid: "9e5905594330410887c8f13761b31e2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:41.999272 22572 catalog_manager.cc:5654] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e5905594330410887c8f13761b31e2f (127.21.127.67). New cstate: current_term: 1 leader_uuid: "9e5905594330410887c8f13761b31e2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:42.015424 22874 raft_consensus.cc:493] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:42.015682 22874 raft_consensus.cc:515] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.016378 22874 leader_election.cc:290] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.016908 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "666c53fa653843e9b3f6ba05a3bb7de7" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f" is_pre_election: true
I20251210 21:03:42.016913 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "666c53fa653843e9b3f6ba05a3bb7de7" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0" is_pre_election: true
I20251210 21:03:42.017108 22823 raft_consensus.cc:2468] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 0.
I20251210 21:03:42.017164 22749 raft_consensus.cc:2468] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 0.
I20251210 21:03:42.017465 22640 leader_election.cc:304] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:42.017706 22874 raft_consensus.cc:2804] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:42.017804 22874 raft_consensus.cc:493] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:42.017868 22874 raft_consensus.cc:3060] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.019708 22874 raft_consensus.cc:515] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.020298 22874 leader_election.cc:290] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Requested vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.020710 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "666c53fa653843e9b3f6ba05a3bb7de7" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f"
I20251210 21:03:42.020704 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "666c53fa653843e9b3f6ba05a3bb7de7" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0"
I20251210 21:03:42.021004 22749 raft_consensus.cc:3060] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.021001 22823 raft_consensus.cc:3060] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.023809 22749 raft_consensus.cc:2468] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:42.023814 22823 raft_consensus.cc:2468] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:42.024309 22640 leader_election.cc:304] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:42.024629 22874 raft_consensus.cc:2804] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:42.024762 22874 raft_consensus.cc:697] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Becoming Leader. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Running, Role: LEADER
I20251210 21:03:42.025043 22874 consensus_queue.cc:237] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.027621 22571 catalog_manager.cc:5654] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "32c994ccef9b4615b5b6df3e8200bde6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:42.055516 22878 raft_consensus.cc:493] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:42.055735 22878 raft_consensus.cc:515] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.056727 22878 leader_election.cc:290] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.063467 22675 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55294cadb3be456fb164afeff111746e" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6" is_pre_election: true
I20251210 21:03:42.063673 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55294cadb3be456fb164afeff111746e" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f" is_pre_election: true
I20251210 21:03:42.063763 22675 raft_consensus.cc:2468] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 0.
I20251210 21:03:42.063870 22823 raft_consensus.cc:2468] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 0.
I20251210 21:03:42.064177 22713 leader_election.cc:304] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:42.064571 22878 raft_consensus.cc:2804] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:42.064671 22878 raft_consensus.cc:493] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:42.064735 22878 raft_consensus.cc:3060] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.066598 22878 raft_consensus.cc:515] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.067132 22878 leader_election.cc:290] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 election: Requested vote from peers 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.067582 22675 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55294cadb3be456fb164afeff111746e" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6"
I20251210 21:03:42.067587 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55294cadb3be456fb164afeff111746e" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f"
I20251210 21:03:42.067780 22675 raft_consensus.cc:3060] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.067807 22823 raft_consensus.cc:3060] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.069754 22675 raft_consensus.cc:2468] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 1.
I20251210 21:03:42.069756 22823 raft_consensus.cc:2468] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 1.
I20251210 21:03:42.070174 22715 leader_election.cc:304] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa6710b99204722af28432ffe9920f0, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:42.070448 22878 raft_consensus.cc:2804] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:42.070817 22878 raft_consensus.cc:697] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Running, Role: LEADER
I20251210 21:03:42.071094 22878 consensus_queue.cc:237] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.073814 22571 catalog_manager.cc:5654] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa6710b99204722af28432ffe9920f0 (127.21.127.66). New cstate: current_term: 1 leader_uuid: "7aa6710b99204722af28432ffe9920f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:42.119983 22878 raft_consensus.cc:493] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:42.120231 22878 raft_consensus.cc:515] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.120918 22878 leader_election.cc:290] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.121457 22675 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11941900d8c24629b869a9bd16ecb96b" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6" is_pre_election: true
I20251210 21:03:42.121479 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11941900d8c24629b869a9bd16ecb96b" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f" is_pre_election: true
I20251210 21:03:42.121662 22675 raft_consensus.cc:2468] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 0.
I20251210 21:03:42.121680 22823 raft_consensus.cc:2468] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 0.
I20251210 21:03:42.122067 22713 leader_election.cc:304] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0; no voters:
I20251210 21:03:42.122330 22878 raft_consensus.cc:2804] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:42.122439 22878 raft_consensus.cc:493] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:42.122543 22878 raft_consensus.cc:3060] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.124473 22878 raft_consensus.cc:515] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.125101 22878 leader_election.cc:290] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 election: Requested vote from peers 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.125486 22675 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11941900d8c24629b869a9bd16ecb96b" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6"
I20251210 21:03:42.125547 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11941900d8c24629b869a9bd16ecb96b" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f"
I20251210 21:03:42.125649 22675 raft_consensus.cc:3060] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.125703 22823 raft_consensus.cc:3060] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.127692 22675 raft_consensus.cc:2468] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 1.
I20251210 21:03:42.127710 22823 raft_consensus.cc:2468] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 1.
I20251210 21:03:42.128206 22713 leader_election.cc:304] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0; no voters:
I20251210 21:03:42.128528 22878 raft_consensus.cc:2804] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:42.128713 22878 raft_consensus.cc:697] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Running, Role: LEADER
I20251210 21:03:42.128968 22878 consensus_queue.cc:237] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.131544 22571 catalog_manager.cc:5654] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa6710b99204722af28432ffe9920f0 (127.21.127.66). New cstate: current_term: 1 leader_uuid: "7aa6710b99204722af28432ffe9920f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:42.133220 22874 raft_consensus.cc:493] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:42.133401 22874 raft_consensus.cc:515] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.133976 22874 leader_election.cc:290] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.134378 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b25e7bfefbf4c7c98791d0cf104a984" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0" is_pre_election: true
I20251210 21:03:42.134578 22749 raft_consensus.cc:2468] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 0.
I20251210 21:03:42.134714 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b25e7bfefbf4c7c98791d0cf104a984" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f" is_pre_election: true
I20251210 21:03:42.134951 22823 raft_consensus.cc:2468] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 0.
I20251210 21:03:42.134956 22641 leader_election.cc:304] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 7aa6710b99204722af28432ffe9920f0; no voters:
I20251210 21:03:42.135274 22874 raft_consensus.cc:2804] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:42.135383 22874 raft_consensus.cc:493] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:42.135505 22874 raft_consensus.cc:3060] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.137272 22874 raft_consensus.cc:515] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.137778 22874 leader_election.cc:290] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Requested vote from peers 7aa6710b99204722af28432ffe9920f0 (127.21.127.66:44985), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.138193 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b25e7bfefbf4c7c98791d0cf104a984" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5905594330410887c8f13761b31e2f"
I20251210 21:03:42.138195 22749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b25e7bfefbf4c7c98791d0cf104a984" candidate_uuid: "32c994ccef9b4615b5b6df3e8200bde6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa6710b99204722af28432ffe9920f0"
I20251210 21:03:42.138402 22749 raft_consensus.cc:3060] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.138411 22823 raft_consensus.cc:3060] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:42.141212 22823 raft_consensus.cc:2468] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:42.141218 22749 raft_consensus.cc:2468] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c994ccef9b4615b5b6df3e8200bde6 in term 1.
I20251210 21:03:42.141639 22640 leader_election.cc:304] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c994ccef9b4615b5b6df3e8200bde6, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:42.141913 22874 raft_consensus.cc:2804] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:42.142086 22874 raft_consensus.cc:697] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Becoming Leader. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Running, Role: LEADER
I20251210 21:03:42.142314 22874 consensus_queue.cc:237] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.145051 22571 catalog_manager.cc:5654] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "32c994ccef9b4615b5b6df3e8200bde6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:42.161909 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:42.164845 22918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:42.165232 22919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:42.166013 22921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:42.166576 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:42.167023 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:42.167121 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:42.167182 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400622167165 us; error 0 us; skew 500 ppm
I20251210 21:03:42.167356 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:42.168635 22013 webserver.cc:492] Webserver started at http://127.21.127.68:40029/ using document root <none> and password file <none>
I20251210 21:03:42.168944 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:42.169025 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:42.169183 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:42.169852 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "68d886687df14daf9b91ca3a7b69cdbd"
format_stamp: "Formatted at 2025-12-10 21:03:42 on dist-test-slave-9pq8"
I20251210 21:03:42.172446 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251210 21:03:42.174139 22926 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:42.174532 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:42.174642 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "68d886687df14daf9b91ca3a7b69cdbd"
format_stamp: "Formatted at 2025-12-10 21:03:42 on dist-test-slave-9pq8"
I20251210 21:03:42.174747 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:42.196861 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:42.197465 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:42.198017 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:42.198858 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:42.198940 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:42.199023 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:42.199076 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:42.215318 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:43175
I20251210 21:03:42.215358 22988 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:43175 every 8 connection(s)
I20251210 21:03:42.230247 22989 heartbeater.cc:344] Connected to a master server at 127.21.127.126:43953
I20251210 21:03:42.230459 22989 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:42.231204 22989 heartbeater.cc:507] Master 127.21.127.126:43953 requested a full tablet report, sending...
I20251210 21:03:42.232555 22571 ts_manager.cc:194] Registered new tserver with Master: 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68:43175)
I20251210 21:03:42.233924 22571 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39250
I20251210 21:03:42.377679 22874 consensus_queue.cc:1048] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.383911 22889 consensus_queue.cc:1048] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.387001 22626 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:03:42.388734 22908 consensus_queue.cc:1048] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.395215 22908 consensus_queue.cc:1048] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.395370 22877 consensus_queue.cc:1048] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.410827 22877 consensus_queue.cc:1048] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.414104 22675 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244"
dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6"
mode: GRACEFUL
new_leader_uuid: "7aa6710b99204722af28432ffe9920f0"
from {username='slave'} at 127.0.0.1:43644
I20251210 21:03:42.414373 22675 raft_consensus.cc:606] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Received request to transfer leadership to TS 7aa6710b99204722af28432ffe9920f0
I20251210 21:03:42.414855 22626 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:03:42.415146 22626 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:03:42.463505 22889 consensus_queue.cc:1048] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.466382 22874 raft_consensus.cc:993] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6: : Instructing follower 7aa6710b99204722af28432ffe9920f0 to start an election
I20251210 21:03:42.466584 22874 raft_consensus.cc:1081] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Signalling peer 7aa6710b99204722af28432ffe9920f0 to start an election
I20251210 21:03:42.468019 22750 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244"
dest_uuid: "7aa6710b99204722af28432ffe9920f0"
from {username='slave'} at 127.0.0.1:33508
I20251210 21:03:42.468295 22750 raft_consensus.cc:493] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:03:42.468425 22750 raft_consensus.cc:3060] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:03:42.468997 22900 consensus_queue.cc:1048] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.472540 22750 raft_consensus.cc:515] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.473595 22675 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32c994ccef9b4615b5b6df3e8200bde6"
I20251210 21:03:42.473822 22675 raft_consensus.cc:3055] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:03:42.473913 22675 raft_consensus.cc:740] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 32c994ccef9b4615b5b6df3e8200bde6, State: Running, Role: LEADER
I20251210 21:03:42.474097 22823 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bacefe14d5e64db592b052b9f269c244" candidate_uuid: "7aa6710b99204722af28432ffe9920f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e5905594330410887c8f13761b31e2f"
I20251210 21:03:42.474236 22750 leader_election.cc:290] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 2 election: Requested vote from peers 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235), 9e5905594330410887c8f13761b31e2f (127.21.127.67:46615)
I20251210 21:03:42.474279 22823 raft_consensus.cc:3060] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:03:42.474243 22675 consensus_queue.cc:260] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.474658 22675 raft_consensus.cc:3060] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:03:42.476316 22749 raft_consensus.cc:1240] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 2 FOLLOWER]: Rejecting Update request from peer 32c994ccef9b4615b5b6df3e8200bde6 for earlier term 1. Current term is 2. Ops: []
I20251210 21:03:42.477504 22823 raft_consensus.cc:2468] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 2.
I20251210 21:03:42.477918 22675 raft_consensus.cc:2468] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa6710b99204722af28432ffe9920f0 in term 2.
I20251210 21:03:42.477960 22715 leader_election.cc:304] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa6710b99204722af28432ffe9920f0, 9e5905594330410887c8f13761b31e2f; no voters:
I20251210 21:03:42.478257 22878 raft_consensus.cc:2804] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:03:42.478394 22878 raft_consensus.cc:697] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 2 LEADER]: Becoming Leader. State: Replica: 7aa6710b99204722af28432ffe9920f0, State: Running, Role: LEADER
I20251210 21:03:42.478657 22878 consensus_queue.cc:237] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } }
I20251210 21:03:42.480679 22571 catalog_manager.cc:5654] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 reported cstate change: term changed from 1 to 2, leader changed from 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65) to 7aa6710b99204722af28432ffe9920f0 (127.21.127.66). New cstate: current_term: 2 leader_uuid: "7aa6710b99204722af28432ffe9920f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:42.523391 22878 consensus_queue.cc:1048] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.528566 22914 consensus_queue.cc:1048] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.579746 22874 consensus_queue.cc:1048] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.583894 22900 consensus_queue.cc:1048] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.597159 22878 consensus_queue.cc:1048] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.601832 23012 consensus_queue.cc:1048] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.606871 22900 consensus_queue.cc:1048] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.610842 22900 consensus_queue.cc:1048] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:42.886765 22675 raft_consensus.cc:1275] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 2 FOLLOWER]: Refusing update from remote peer 7aa6710b99204722af28432ffe9920f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:03:42.887310 22914 consensus_queue.cc:1048] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:42.890489 22823 raft_consensus.cc:1275] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 2 FOLLOWER]: Refusing update from remote peer 7aa6710b99204722af28432ffe9920f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:03:42.890985 22914 consensus_queue.cc:1048] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:43.236089 22989 heartbeater.cc:499] Master 127.21.127.126:43953 was elected leader, sending a full tablet report...
I20251210 21:03:44.415632 22626 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:03:44.417068 22626 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:03:44.417229 22626 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:03:46.417758 22626 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:03:46.419185 22626 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:03:46.419373 22626 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:03:47.399127 22823 consensus_queue.cc:237] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } }
I20251210 21:03:47.408630 22675 raft_consensus.cc:1275] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Refusing update from remote peer 9e5905594330410887c8f13761b31e2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:47.409404 22907 consensus_queue.cc:1048] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251210 21:03:47.416077 22787 consensus_peers.cc:597] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f -> Peer 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68:43175): Couldn't send request to peer 68d886687df14daf9b91ca3a7b69cdbd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfb1b71eb54349289ce17d8812199f90. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:03:47.416184 22749 raft_consensus.cc:1275] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Refusing update from remote peer 9e5905594330410887c8f13761b31e2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:47.415868 23025 raft_consensus.cc:2955] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.416730 23021 consensus_queue.cc:1048] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:47.416730 22675 raft_consensus.cc:2955] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.418694 22749 raft_consensus.cc:2955] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.422346 22571 catalog_manager.cc:5654] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 reported cstate change: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "9e5905594330410887c8f13761b31e2f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.434356 22749 consensus_queue.cc:237] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } attrs { replace: true } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } }
I20251210 21:03:47.437517 22823 raft_consensus.cc:1275] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Refusing update from remote peer 7aa6710b99204722af28432ffe9920f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:47.438078 23033 consensus_queue.cc:1048] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:47.441871 22675 raft_consensus.cc:1275] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Refusing update from remote peer 7aa6710b99204722af28432ffe9920f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:47.443595 23033 consensus_queue.cc:1048] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:47.444470 23033 raft_consensus.cc:2955] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } attrs { replace: true } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.445395 22823 raft_consensus.cc:2955] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } attrs { replace: true } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
W20251210 21:03:47.448282 22713 consensus_peers.cc:597] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 -> Peer 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68:43175): Couldn't send request to peer 68d886687df14daf9b91ca3a7b69cdbd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55294cadb3be456fb164afeff111746e. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:03:47.450304 22573 catalog_manager.cc:5654] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 reported cstate change: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "7aa6710b99204722af28432ffe9920f0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:47.455397 22675 raft_consensus.cc:2955] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } attrs { replace: true } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.461746 22675 consensus_queue.cc:237] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } }
I20251210 21:03:47.464479 22749 raft_consensus.cc:1275] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Refusing update from remote peer 32c994ccef9b4615b5b6df3e8200bde6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:47.465087 23016 consensus_queue.cc:1048] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:47.468466 22823 raft_consensus.cc:1275] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Refusing update from remote peer 32c994ccef9b4615b5b6df3e8200bde6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:47.469039 23016 consensus_queue.cc:1048] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:47.472761 23016 raft_consensus.cc:2955] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.480615 22749 raft_consensus.cc:2955] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.482604 22675 consensus_queue.cc:237] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } }
W20251210 21:03:47.488250 22638 consensus_peers.cc:597] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 -> Peer 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68:43175): Couldn't send request to peer 68d886687df14daf9b91ca3a7b69cdbd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b25e7bfefbf4c7c98791d0cf104a984. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:03:47.490337 22573 catalog_manager.cc:5654] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 reported cstate change: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "32c994ccef9b4615b5b6df3e8200bde6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.492386 22749 raft_consensus.cc:1275] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Refusing update from remote peer 32c994ccef9b4615b5b6df3e8200bde6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251210 21:03:47.492808 22638 consensus_peers.cc:597] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 -> Peer 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68:43175): Couldn't send request to peer 68d886687df14daf9b91ca3a7b69cdbd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 044baf974d174b988cdf96de51b766cf. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:03:47.493036 23044 consensus_queue.cc:1048] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:47.495039 22824 raft_consensus.cc:1275] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Refusing update from remote peer 32c994ccef9b4615b5b6df3e8200bde6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:47.494791 22823 raft_consensus.cc:2955] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.495957 23040 consensus_queue.cc:1048] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:47.498540 23040 raft_consensus.cc:2955] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.499755 22749 raft_consensus.cc:2955] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.503916 22573 catalog_manager.cc:5654] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 reported cstate change: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "32c994ccef9b4615b5b6df3e8200bde6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:47.504567 22824 raft_consensus.cc:2955] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 68d886687df14daf9b91ca3a7b69cdbd (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "32c994ccef9b4615b5b6df3e8200bde6" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 37235 } } peers { permanent_uuid: "7aa6710b99204722af28432ffe9920f0" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 44985 } attrs { replace: true } } peers { permanent_uuid: "9e5905594330410887c8f13761b31e2f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 46615 } } peers { permanent_uuid: "68d886687df14daf9b91ca3a7b69cdbd" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43175 } attrs { promote: true } } }
I20251210 21:03:47.731339 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
W20251210 21:03:47.733834 22557 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20251210 21:03:47.736577 22621 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:03:47.736860 22621 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251210 21:03:47.741240 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:47.741541 22013 tablet_replica.cc:333] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
I20251210 21:03:47.741760 22013 raft_consensus.cc:2243] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.741957 22013 raft_consensus.cc:2272] T 11941900d8c24629b869a9bd16ecb96b P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.742527 22013 tablet_replica.cc:333] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
W20251210 21:03:47.742678 22621 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251210 21:03:47.742714 22013 raft_consensus.cc:2243] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251210 21:03:47.742770 22621 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251210 21:03:47.742897 22013 raft_consensus.cc:2272] T 01df683a26744048a50073eafe485b5a P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.743479 22013 tablet_replica.cc:333] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
I20251210 21:03:47.743652 22013 raft_consensus.cc:2243] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.744141 22013 raft_consensus.cc:2272] T 3b25e7bfefbf4c7c98791d0cf104a984 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.744786 22013 tablet_replica.cc:333] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
I20251210 21:03:47.744947 22013 raft_consensus.cc:2243] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.745215 22013 raft_consensus.cc:2272] T 666c53fa653843e9b3f6ba05a3bb7de7 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.745734 22013 tablet_replica.cc:333] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
I20251210 21:03:47.745872 22013 raft_consensus.cc:2243] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.746227 22013 raft_consensus.cc:2272] T 044baf974d174b988cdf96de51b766cf P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.746871 22013 tablet_replica.cc:333] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
I20251210 21:03:47.747022 22013 raft_consensus.cc:2243] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.747210 22013 raft_consensus.cc:2272] T cfb1b71eb54349289ce17d8812199f90 P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.748055 22013 tablet_replica.cc:333] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
I20251210 21:03:47.748204 22013 raft_consensus.cc:2243] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.748405 22013 raft_consensus.cc:2272] T bacefe14d5e64db592b052b9f269c244 P 32c994ccef9b4615b5b6df3e8200bde6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.748955 22013 tablet_replica.cc:333] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6: stopping tablet replica
I20251210 21:03:47.749123 22013 raft_consensus.cc:2243] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.749316 22013 raft_consensus.cc:2272] T 55294cadb3be456fb164afeff111746e P 32c994ccef9b4615b5b6df3e8200bde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.766584 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:03:47.771354 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
W20251210 21:03:47.775287 22621 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:03:47.775409 22621 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251210 21:03:47.781648 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20251210 21:03:47.781630 22787 consensus_peers.cc:597] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f -> Peer 32c994ccef9b4615b5b6df3e8200bde6 (127.21.127.65:37235): Couldn't send request to peer 32c994ccef9b4615b5b6df3e8200bde6. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:37235: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:03:47.781906 22013 tablet_replica.cc:333] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.782133 22013 raft_consensus.cc:2243] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.782490 22013 raft_consensus.cc:2272] T 11941900d8c24629b869a9bd16ecb96b P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.783095 22013 tablet_replica.cc:333] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.783241 22013 raft_consensus.cc:2243] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.783432 22013 raft_consensus.cc:2272] T 3b25e7bfefbf4c7c98791d0cf104a984 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.784236 22013 tablet_replica.cc:333] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.784402 22013 raft_consensus.cc:2243] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.784570 22013 raft_consensus.cc:2272] T 01df683a26744048a50073eafe485b5a P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.785041 22013 tablet_replica.cc:333] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.785245 22013 raft_consensus.cc:2243] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.785567 22013 raft_consensus.cc:2272] T 55294cadb3be456fb164afeff111746e P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.786228 22013 tablet_replica.cc:333] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.786377 22013 raft_consensus.cc:2243] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.786604 22013 raft_consensus.cc:2272] T bacefe14d5e64db592b052b9f269c244 P 7aa6710b99204722af28432ffe9920f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.787070 22013 tablet_replica.cc:333] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.787213 22013 raft_consensus.cc:2243] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.787367 22013 raft_consensus.cc:2272] T 666c53fa653843e9b3f6ba05a3bb7de7 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.787853 22013 tablet_replica.cc:333] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.787992 22013 raft_consensus.cc:2243] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.788185 22013 raft_consensus.cc:2272] T cfb1b71eb54349289ce17d8812199f90 P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.788877 22013 tablet_replica.cc:333] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0: stopping tablet replica
I20251210 21:03:47.789040 22013 raft_consensus.cc:2243] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.789238 22013 raft_consensus.cc:2272] T 044baf974d174b988cdf96de51b766cf P 7aa6710b99204722af28432ffe9920f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.806138 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:03:47.811209 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
W20251210 21:03:47.813478 22621 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:03:47.813587 22621 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251210 21:03:47.819804 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:47.820060 22013 tablet_replica.cc:333] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.820248 22013 raft_consensus.cc:2243] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.820420 22013 raft_consensus.cc:2272] T 11941900d8c24629b869a9bd16ecb96b P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.820930 22013 tablet_replica.cc:333] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.821080 22013 raft_consensus.cc:2243] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.821202 22013 raft_consensus.cc:2272] T 666c53fa653843e9b3f6ba05a3bb7de7 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.821596 22013 tablet_replica.cc:333] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.821704 22013 raft_consensus.cc:2243] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.821848 22013 raft_consensus.cc:2272] T 044baf974d174b988cdf96de51b766cf P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.822371 22013 tablet_replica.cc:333] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.822492 22013 raft_consensus.cc:2243] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.822783 22013 raft_consensus.cc:2272] T cfb1b71eb54349289ce17d8812199f90 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.823302 22013 tablet_replica.cc:333] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.823410 22013 raft_consensus.cc:2243] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.823604 22013 raft_consensus.cc:2272] T 3b25e7bfefbf4c7c98791d0cf104a984 P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.824211 22013 tablet_replica.cc:333] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.824331 22013 raft_consensus.cc:2243] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.824542 22013 raft_consensus.cc:2272] T 01df683a26744048a50073eafe485b5a P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.824913 22013 tablet_replica.cc:333] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.825023 22013 raft_consensus.cc:2243] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.825161 22013 raft_consensus.cc:2272] T 55294cadb3be456fb164afeff111746e P 9e5905594330410887c8f13761b31e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.825722 22013 tablet_replica.cc:333] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f: stopping tablet replica
I20251210 21:03:47.825838 22013 raft_consensus.cc:2243] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:47.825948 22013 raft_consensus.cc:2272] T bacefe14d5e64db592b052b9f269c244 P 9e5905594330410887c8f13761b31e2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.831789 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:03:47.835592 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:03:47.844035 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:47.858139 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:03:47.860780 22013 master.cc:561] Master@127.21.127.126:43953 shutting down...
I20251210 21:03:47.868016 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:47.868273 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:47.868392 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5f99f3ef87f64a7faf4d7825a1b1d877: stopping tablet replica
I20251210 21:03:47.872249 22013 master.cc:583] Master@127.21.127.126:43953 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn (7571 ms)
[ RUN ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing
I20251210 21:03:47.887665 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:37635,127.21.127.125:33435,127.21.127.124:35919
I20251210 21:03:47.888302 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:47.891027 23052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:47.891090 23051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:47.891134 23054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:47.891664 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:47.891942 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:47.892009 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:47.892063 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400627892045 us; error 0 us; skew 500 ppm
I20251210 21:03:47.892225 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:47.893355 22013 webserver.cc:492] Webserver started at http://127.21.127.126:40645/ using document root <none> and password file <none>
I20251210 21:03:47.893601 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:47.893692 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:47.893841 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:47.894493 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "cd0782e39dd641eb8551779a94411c1a"
format_stamp: "Formatted at 2025-12-10 21:03:47 on dist-test-slave-9pq8"
I20251210 21:03:47.897010 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:03:47.898654 23059 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:47.899048 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:03:47.899143 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "cd0782e39dd641eb8551779a94411c1a"
format_stamp: "Formatted at 2025-12-10 21:03:47 on dist-test-slave-9pq8"
I20251210 21:03:47.899258 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:47.909547 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:47.910117 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:47.922953 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:37635
I20251210 21:03:47.922991 23110 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:37635 every 8 connection(s)
I20251210 21:03:47.924301 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:03:47.924391 23111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251210 21:03:47.926795 23114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:47.927214 23113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:47.927762 23111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:47.928251 22013 server_base.cc:1047] running on GCE node
W20251210 21:03:47.928555 23116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:47.929330 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:47.929406 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:47.929481 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400627929445 us; error 0 us; skew 500 ppm
I20251210 21:03:47.929620 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:47.930737 22013 webserver.cc:492] Webserver started at http://127.21.127.125:33925/ using document root <none> and password file <none>
I20251210 21:03:47.930948 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:47.931038 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:47.931159 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:47.931804 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root/instance:
uuid: "1cfd9bdacfd04f809ff6e5d686af636c"
format_stamp: "Formatted at 2025-12-10 21:03:47 on dist-test-slave-9pq8"
I20251210 21:03:47.934370 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:03:47.936457 23123 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:47.936921 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:03:47.937058 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root
uuid: "1cfd9bdacfd04f809ff6e5d686af636c"
format_stamp: "Formatted at 2025-12-10 21:03:47 on dist-test-slave-9pq8"
I20251210 21:03:47.937199 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:47.937942 23111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } has no permanent_uuid. Determining permanent_uuid...
W20251210 21:03:47.940063 23111 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.127.125:33435: Network error: Client connection negotiation failed: client connection to 127.21.127.125:33435: connect: Connection refused (error 111)
I20251210 21:03:47.947718 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:47.948328 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:47.960510 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.125:33435
I20251210 21:03:47.960551 23175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.125:33435 every 8 connection(s)
I20251210 21:03:47.961850 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:03:47.961875 23176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251210 21:03:47.964331 23178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:47.964332 23179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:47.965207 23176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
W20251210 21:03:47.965742 23181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:47.966252 23111 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } attempt: 1
I20251210 21:03:47.966480 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:47.966790 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:47.966878 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:47.966953 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400627966911 us; error 0 us; skew 500 ppm
I20251210 21:03:47.967171 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:47.969221 22013 webserver.cc:492] Webserver started at http://127.21.127.124:42713/ using document root <none> and password file <none>
I20251210 21:03:47.969491 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:47.969604 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:47.969755 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:47.970582 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root/instance:
uuid: "a1d81542db044c5da6ff2167351201fe"
format_stamp: "Formatted at 2025-12-10 21:03:47 on dist-test-slave-9pq8"
I20251210 21:03:47.972990 23176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:47.973481 23111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:47.974046 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
W20251210 21:03:47.975700 23111 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.127.124:35919: Network error: Client connection negotiation failed: client connection to 127.21.127.124:35919: connect: Connection refused (error 111)
I20251210 21:03:47.976275 23188 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:47.976639 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:03:47.976742 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root
uuid: "a1d81542db044c5da6ff2167351201fe"
format_stamp: "Formatted at 2025-12-10 21:03:47 on dist-test-slave-9pq8"
I20251210 21:03:47.976903 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:47.979990 23176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } has no permanent_uuid. Determining permanent_uuid...
W20251210 21:03:47.982324 23176 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.127.124:35919: Network error: Client connection negotiation failed: client connection to 127.21.127.124:35919: connect: Connection refused (error 111)
I20251210 21:03:47.999116 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:47.999758 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:48.011161 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.124:35919
I20251210 21:03:48.011201 23240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.124:35919 every 8 connection(s)
I20251210 21:03:48.012106 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:03:48.012588 23241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:48.014829 23241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:48.016889 23111 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } attempt: 1
I20251210 21:03:48.022013 23241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:48.025516 23176 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } attempt: 1
I20251210 21:03:48.026309 23111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a: Bootstrap starting.
I20251210 21:03:48.028192 23111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:48.028590 23241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } has no permanent_uuid. Determining permanent_uuid...
I20251210 21:03:48.030725 23111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a: No bootstrap required, opened a new log
I20251210 21:03:48.031616 23111 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.031848 23111 raft_consensus.cc:385] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:48.031944 23111 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0782e39dd641eb8551779a94411c1a, State: Initialized, Role: FOLLOWER
I20251210 21:03:48.032270 23111 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.032915 23247 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.033195 23247 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:48.033238 23111 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:48.035151 23176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: Bootstrap starting.
I20251210 21:03:48.037086 23176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:48.038793 23241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: Bootstrap starting.
I20251210 21:03:48.040652 23241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:48.042806 23241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: No bootstrap required, opened a new log
I20251210 21:03:48.043707 23241 raft_consensus.cc:359] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.043943 23241 raft_consensus.cc:385] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:48.044029 23241 raft_consensus.cc:740] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d81542db044c5da6ff2167351201fe, State: Initialized, Role: FOLLOWER
I20251210 21:03:48.044291 23241 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.044425 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251210 21:03:48.045799 23260 catalog_manager.cc:1568] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251210 21:03:48.045917 23260 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251210 21:03:48.047782 23261 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.047974 23261 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:48.048086 23241 sys_catalog.cc:565] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:48.048447 23176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: No bootstrap required, opened a new log
I20251210 21:03:48.049134 23176 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.049312 23176 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:48.049391 23176 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cfd9bdacfd04f809ff6e5d686af636c, State: Initialized, Role: FOLLOWER
I20251210 21:03:48.049616 23176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.050151 23264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.050364 23264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:48.050428 23176 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:48.054849 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20251210 21:03:48.055773 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:48.056170 23284 catalog_manager.cc:1568] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251210 21:03:48.056212 23283 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251210 21:03:48.056273 23284 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251210 21:03:48.056303 23283 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251210 21:03:48.058647 23285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:48.058866 23286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:48.059389 23288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:48.060578 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:48.060863 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:48.060935 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:48.060977 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400628060964 us; error 0 us; skew 500 ppm
I20251210 21:03:48.061123 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:48.062276 22013 webserver.cc:492] Webserver started at http://127.21.127.65:44497/ using document root <none> and password file <none>
I20251210 21:03:48.062484 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:48.062547 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:48.062670 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:48.063315 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "f6432d4a30ec469c99dbbdedd5aee089"
format_stamp: "Formatted at 2025-12-10 21:03:48 on dist-test-slave-9pq8"
I20251210 21:03:48.065704 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251210 21:03:48.067167 23293 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.067507 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:48.067628 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "f6432d4a30ec469c99dbbdedd5aee089"
format_stamp: "Formatted at 2025-12-10 21:03:48 on dist-test-slave-9pq8"
I20251210 21:03:48.067799 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:48.093091 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:48.093737 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:48.094305 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:48.095232 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:48.095326 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.095417 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:48.095525 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.110010 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:43615
I20251210 21:03:48.110055 23355 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:43615 every 8 connection(s)
I20251210 21:03:48.125002 23356 heartbeater.cc:344] Connected to a master server at 127.21.127.124:35919
I20251210 21:03:48.125177 23356 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.125319 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:03:48.125653 23356 heartbeater.cc:507] Master 127.21.127.124:35919 requested a full tablet report, sending...
I20251210 21:03:48.126924 23206 ts_manager.cc:194] Registered new tserver with Master: f6432d4a30ec469c99dbbdedd5aee089 (127.21.127.65:43615)
I20251210 21:03:48.127755 23357 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37635
I20251210 21:03:48.127916 23357 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.128449 23357 heartbeater.cc:507] Master 127.21.127.126:37635 requested a full tablet report, sending...
W20251210 21:03:48.129153 23363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:48.129585 23076 ts_manager.cc:194] Registered new tserver with Master: f6432d4a30ec469c99dbbdedd5aee089 (127.21.127.65:43615)
W20251210 21:03:48.130393 23364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:48.132294 23358 heartbeater.cc:344] Connected to a master server at 127.21.127.125:33435
I20251210 21:03:48.132449 23358 heartbeater.cc:461] Registering TS with master...
W20251210 21:03:48.132483 23366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:48.132824 23358 heartbeater.cc:507] Master 127.21.127.125:33435 requested a full tablet report, sending...
I20251210 21:03:48.133087 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:48.133380 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:48.133468 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:48.133560 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400628133511 us; error 0 us; skew 500 ppm
I20251210 21:03:48.133836 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:48.133894 23141 ts_manager.cc:194] Registered new tserver with Master: f6432d4a30ec469c99dbbdedd5aee089 (127.21.127.65:43615)
I20251210 21:03:48.135052 22013 webserver.cc:492] Webserver started at http://127.21.127.66:41821/ using document root <none> and password file <none>
I20251210 21:03:48.135257 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:48.135353 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:48.135506 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:48.136145 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "6556ccbe398347e9a7d4c58486d6508f"
format_stamp: "Formatted at 2025-12-10 21:03:48 on dist-test-slave-9pq8"
I20251210 21:03:48.138567 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:03:48.140312 23371 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.140782 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:03:48.140893 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "6556ccbe398347e9a7d4c58486d6508f"
format_stamp: "Formatted at 2025-12-10 21:03:48 on dist-test-slave-9pq8"
I20251210 21:03:48.141028 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:48.149866 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:48.150418 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:48.151016 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:48.152062 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:48.152133 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.152240 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:48.152292 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.167747 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:36625
I20251210 21:03:48.167788 23433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:36625 every 8 connection(s)
I20251210 21:03:48.182158 23435 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37635
I20251210 21:03:48.182371 23435 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.182886 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:03:48.182927 23435 heartbeater.cc:507] Master 127.21.127.126:37635 requested a full tablet report, sending...
I20251210 21:03:48.183583 23434 heartbeater.cc:344] Connected to a master server at 127.21.127.124:35919
I20251210 21:03:48.183696 23434 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.184010 23076 ts_manager.cc:194] Registered new tserver with Master: 6556ccbe398347e9a7d4c58486d6508f (127.21.127.66:36625)
I20251210 21:03:48.184152 23434 heartbeater.cc:507] Master 127.21.127.124:35919 requested a full tablet report, sending...
I20251210 21:03:48.185108 23206 ts_manager.cc:194] Registered new tserver with Master: 6556ccbe398347e9a7d4c58486d6508f (127.21.127.66:36625)
I20251210 21:03:48.185364 23436 heartbeater.cc:344] Connected to a master server at 127.21.127.125:33435
I20251210 21:03:48.185501 23436 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.185897 23436 heartbeater.cc:507] Master 127.21.127.125:33435 requested a full tablet report, sending...
W20251210 21:03:48.186439 23441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:48.186866 23141 ts_manager.cc:194] Registered new tserver with Master: 6556ccbe398347e9a7d4c58486d6508f (127.21.127.66:36625)
W20251210 21:03:48.187891 23442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:48.188737 23444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:48.189003 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:48.189260 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:48.189337 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:48.189392 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400628189375 us; error 0 us; skew 500 ppm
I20251210 21:03:48.189564 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:48.190742 22013 webserver.cc:492] Webserver started at http://127.21.127.67:39377/ using document root <none> and password file <none>
I20251210 21:03:48.190963 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:48.191037 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:48.191172 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:48.191825 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "250678d1c696453ca9eb750d0325360f"
format_stamp: "Formatted at 2025-12-10 21:03:48 on dist-test-slave-9pq8"
I20251210 21:03:48.194240 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:03:48.195904 23449 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.196214 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:48.196334 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "250678d1c696453ca9eb750d0325360f"
format_stamp: "Formatted at 2025-12-10 21:03:48 on dist-test-slave-9pq8"
I20251210 21:03:48.196449 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:48.217077 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:48.217692 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:48.218432 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:48.219352 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:48.219468 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.219568 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:48.219641 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:48.234661 23247 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:48.234978 23247 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.235782 23247 leader_election.cc:290] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cfd9bdacfd04f809ff6e5d686af636c (127.21.127.125:33435), a1d81542db044c5da6ff2167351201fe (127.21.127.124:35919)
I20251210 21:03:48.236320 23151 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd0782e39dd641eb8551779a94411c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" is_pre_election: true
I20251210 21:03:48.236356 23216 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd0782e39dd641eb8551779a94411c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d81542db044c5da6ff2167351201fe" is_pre_election: true
I20251210 21:03:48.236676 23216 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0782e39dd641eb8551779a94411c1a in term 0.
I20251210 21:03:48.236673 23151 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0782e39dd641eb8551779a94411c1a in term 0.
I20251210 21:03:48.237203 23060 leader_election.cc:304] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1d81542db044c5da6ff2167351201fe, cd0782e39dd641eb8551779a94411c1a; no voters:
I20251210 21:03:48.237493 23247 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:48.237623 23247 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:48.237726 23247 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:48.238076 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:34229
I20251210 21:03:48.238118 23511 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:34229 every 8 connection(s)
I20251210 21:03:48.240571 23247 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.241240 23247 leader_election.cc:290] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [CANDIDATE]: Term 1 election: Requested vote from peers 1cfd9bdacfd04f809ff6e5d686af636c (127.21.127.125:33435), a1d81542db044c5da6ff2167351201fe (127.21.127.124:35919)
I20251210 21:03:48.241801 23151 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd0782e39dd641eb8551779a94411c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cfd9bdacfd04f809ff6e5d686af636c"
I20251210 21:03:48.242007 23151 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:48.244302 23151 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0782e39dd641eb8551779a94411c1a in term 1.
I20251210 21:03:48.244557 23216 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd0782e39dd641eb8551779a94411c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d81542db044c5da6ff2167351201fe"
I20251210 21:03:48.244760 23216 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:48.244768 23060 leader_election.cc:304] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cfd9bdacfd04f809ff6e5d686af636c, cd0782e39dd641eb8551779a94411c1a; no voters:
I20251210 21:03:48.245229 23247 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:48.247831 23216 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0782e39dd641eb8551779a94411c1a in term 1.
I20251210 21:03:48.257506 23247 raft_consensus.cc:697] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 1 LEADER]: Becoming Leader. State: Replica: cd0782e39dd641eb8551779a94411c1a, State: Running, Role: LEADER
I20251210 21:03:48.257696 23513 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37635
I20251210 21:03:48.257901 23513 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.258128 23512 heartbeater.cc:344] Connected to a master server at 127.21.127.124:35919
I20251210 21:03:48.258342 23512 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.258436 23247 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:48.259030 23513 heartbeater.cc:507] Master 127.21.127.126:37635 requested a full tablet report, sending...
I20251210 21:03:48.259387 23512 heartbeater.cc:507] Master 127.21.127.124:35919 requested a full tablet report, sending...
I20251210 21:03:48.260423 23206 ts_manager.cc:194] Registered new tserver with Master: 250678d1c696453ca9eb750d0325360f (127.21.127.67:34229)
I20251210 21:03:48.260857 23076 ts_manager.cc:194] Registered new tserver with Master: 250678d1c696453ca9eb750d0325360f (127.21.127.67:34229)
I20251210 21:03:48.260941 23517 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0782e39dd641eb8551779a94411c1a. Latest consensus state: current_term: 1 leader_uuid: "cd0782e39dd641eb8551779a94411c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.261240 23517 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:48.264050 23520 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:03:48.265618 23514 heartbeater.cc:344] Connected to a master server at 127.21.127.125:33435
I20251210 21:03:48.265795 23514 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:48.265980 23520 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:03:48.266328 23514 heartbeater.cc:507] Master 127.21.127.125:33435 requested a full tablet report, sending...
I20251210 21:03:48.267331 23141 ts_manager.cc:194] Registered new tserver with Master: 250678d1c696453ca9eb750d0325360f (127.21.127.67:34229)
I20251210 21:03:48.268276 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.008118843s
I20251210 21:03:48.269582 23216 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Refusing update from remote peer cd0782e39dd641eb8551779a94411c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:48.270316 23151 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Refusing update from remote peer cd0782e39dd641eb8551779a94411c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:03:48.270267 23247 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:48.270826 23247 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:48.275980 23261 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0782e39dd641eb8551779a94411c1a. Latest consensus state: current_term: 1 leader_uuid: "cd0782e39dd641eb8551779a94411c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.276196 23261 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:48.277452 23264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0782e39dd641eb8551779a94411c1a. Latest consensus state: current_term: 1 leader_uuid: "cd0782e39dd641eb8551779a94411c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.277695 23264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:48.279014 23261 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd0782e39dd641eb8551779a94411c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.279244 23261 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:48.279408 23517 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd0782e39dd641eb8551779a94411c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.279603 23517 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:48.279973 23520 catalog_manager.cc:1357] Generated new cluster ID: b8851b687f4840fd806e6711cdc4ddfe
I20251210 21:03:48.280059 23520 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:03:48.280333 23247 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd0782e39dd641eb8551779a94411c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.280578 23247 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:48.281703 23264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd0782e39dd641eb8551779a94411c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:48.281901 23264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [sys.catalog]: This master's current role is: FOLLOWER
I20251210 21:03:48.300410 23520 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:03:48.301436 23520 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:03:48.309796 23520 catalog_manager.cc:6027] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a: Generated new TSK 0
I20251210 21:03:48.310187 23520 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:03:49.057647 23284 catalog_manager.cc:1269] Loaded cluster ID: b8851b687f4840fd806e6711cdc4ddfe
I20251210 21:03:49.057732 23283 catalog_manager.cc:1269] Loaded cluster ID: b8851b687f4840fd806e6711cdc4ddfe
I20251210 21:03:49.057821 23284 catalog_manager.cc:1562] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: loading cluster ID for follower catalog manager: success
I20251210 21:03:49.057855 23283 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: loading cluster ID for follower catalog manager: success
I20251210 21:03:49.059907 23284 catalog_manager.cc:1584] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: acquiring CA information for follower catalog manager: success
I20251210 21:03:49.059928 23283 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: acquiring CA information for follower catalog manager: success
I20251210 21:03:49.061040 23283 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251210 21:03:49.061038 23284 catalog_manager.cc:1612] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251210 21:03:49.132340 23076 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33154
I20251210 21:03:49.186817 23076 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33170
I20251210 21:03:49.263171 23076 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33174
I20251210 21:03:50.044493 23259 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:03:50.133581 23357 heartbeater.cc:499] Master 127.21.127.126:37635 was elected leader, sending a full tablet report...
I20251210 21:03:50.188153 23435 heartbeater.cc:499] Master 127.21.127.126:37635 was elected leader, sending a full tablet report...
I20251210 21:03:50.264353 23513 heartbeater.cc:499] Master 127.21.127.126:37635 was elected leader, sending a full tablet report...
I20251210 21:03:50.424778 22013 test_util.cc:276] Using random seed: 1568044190
I20251210 21:03:50.457101 23076 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33184:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251210 21:03:50.498654 23477 tablet_service.cc:1505] Processing CreateTablet for tablet 390e5d5b6ee747c0a0621217f2f07c54 (DEFAULT_TABLE table=test-workload [id=2b057417f5184bbeaae9acc10d9d7345]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251210 21:03:50.499326 23477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 390e5d5b6ee747c0a0621217f2f07c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:50.499621 23321 tablet_service.cc:1505] Processing CreateTablet for tablet 0422b8cfc21f45c5a96bf18b873bf9a8 (DEFAULT_TABLE table=test-workload [id=2b057417f5184bbeaae9acc10d9d7345]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251210 21:03:50.500051 23321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0422b8cfc21f45c5a96bf18b873bf9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:50.504868 23560 tablet_bootstrap.cc:492] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f: Bootstrap starting.
I20251210 21:03:50.505391 23561 tablet_bootstrap.cc:492] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089: Bootstrap starting.
I20251210 21:03:50.505817 23399 tablet_service.cc:1505] Processing CreateTablet for tablet b2536231db684e538d76e8638bb12647 (DEFAULT_TABLE table=test-workload [id=2b057417f5184bbeaae9acc10d9d7345]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251210 21:03:50.506376 23399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2536231db684e538d76e8638bb12647. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:50.507028 23560 tablet_bootstrap.cc:654] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:50.508428 23561 tablet_bootstrap.cc:654] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:50.508920 23560 tablet_bootstrap.cc:492] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f: No bootstrap required, opened a new log
I20251210 21:03:50.509075 23560 ts_tablet_manager.cc:1403] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:03:50.509869 23560 raft_consensus.cc:359] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250678d1c696453ca9eb750d0325360f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34229 } }
I20251210 21:03:50.510027 23560 raft_consensus.cc:385] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:50.510159 23560 raft_consensus.cc:740] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 250678d1c696453ca9eb750d0325360f, State: Initialized, Role: FOLLOWER
I20251210 21:03:50.510419 23561 tablet_bootstrap.cc:492] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089: No bootstrap required, opened a new log
I20251210 21:03:50.510412 23560 consensus_queue.cc:260] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250678d1c696453ca9eb750d0325360f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34229 } }
I20251210 21:03:50.510632 23561 ts_tablet_manager.cc:1403] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251210 21:03:50.510672 23560 raft_consensus.cc:399] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:50.510777 23560 raft_consensus.cc:493] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:50.510905 23560 raft_consensus.cc:3060] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:50.511463 23561 raft_consensus.cc:359] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6432d4a30ec469c99dbbdedd5aee089" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43615 } }
I20251210 21:03:50.511721 23561 raft_consensus.cc:385] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:50.511862 23561 raft_consensus.cc:740] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6432d4a30ec469c99dbbdedd5aee089, State: Initialized, Role: FOLLOWER
I20251210 21:03:50.512581 23561 consensus_queue.cc:260] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6432d4a30ec469c99dbbdedd5aee089" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43615 } }
I20251210 21:03:50.512796 23561 raft_consensus.cc:399] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:50.512887 23561 raft_consensus.cc:493] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:50.513003 23561 raft_consensus.cc:3060] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:50.513257 23560 raft_consensus.cc:515] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250678d1c696453ca9eb750d0325360f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34229 } }
I20251210 21:03:50.513474 23560 leader_election.cc:304] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 250678d1c696453ca9eb750d0325360f; no voters:
I20251210 21:03:50.513969 23560 leader_election.cc:290] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:50.514182 23565 raft_consensus.cc:2804] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:50.515849 23561 raft_consensus.cc:515] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6432d4a30ec469c99dbbdedd5aee089" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43615 } }
I20251210 21:03:50.516145 23561 leader_election.cc:304] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6432d4a30ec469c99dbbdedd5aee089; no voters:
I20251210 21:03:50.517428 23568 raft_consensus.cc:2804] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:50.517592 23561 leader_election.cc:290] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:50.521513 23564 tablet_bootstrap.cc:492] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f: Bootstrap starting.
I20251210 21:03:50.524842 23564 tablet_bootstrap.cc:654] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:50.525729 23568 raft_consensus.cc:697] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 1 LEADER]: Becoming Leader. State: Replica: f6432d4a30ec469c99dbbdedd5aee089, State: Running, Role: LEADER
I20251210 21:03:50.526376 23568 consensus_queue.cc:237] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6432d4a30ec469c99dbbdedd5aee089" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43615 } }
I20251210 21:03:50.527294 23560 ts_tablet_manager.cc:1434] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20251210 21:03:50.528255 23561 ts_tablet_manager.cc:1434] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089: Time spent starting tablet: real 0.017s user 0.005s sys 0.005s
I20251210 21:03:50.528250 23565 raft_consensus.cc:697] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 1 LEADER]: Becoming Leader. State: Replica: 250678d1c696453ca9eb750d0325360f, State: Running, Role: LEADER
I20251210 21:03:50.529153 23565 consensus_queue.cc:237] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250678d1c696453ca9eb750d0325360f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34229 } }
I20251210 21:03:50.532156 23076 catalog_manager.cc:5654] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f reported cstate change: term changed from 0 to 1, leader changed from <none> to 250678d1c696453ca9eb750d0325360f (127.21.127.67). New cstate: current_term: 1 leader_uuid: "250678d1c696453ca9eb750d0325360f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250678d1c696453ca9eb750d0325360f" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34229 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:50.536185 23075 catalog_manager.cc:5654] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6432d4a30ec469c99dbbdedd5aee089 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "f6432d4a30ec469c99dbbdedd5aee089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6432d4a30ec469c99dbbdedd5aee089" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43615 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:50.549715 23564 tablet_bootstrap.cc:492] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f: No bootstrap required, opened a new log
I20251210 21:03:50.550011 23564 ts_tablet_manager.cc:1403] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f: Time spent bootstrapping tablet: real 0.029s user 0.014s sys 0.009s
I20251210 21:03:50.550976 23564 raft_consensus.cc:359] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556ccbe398347e9a7d4c58486d6508f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 36625 } }
I20251210 21:03:50.551259 23564 raft_consensus.cc:385] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:50.551407 23564 raft_consensus.cc:740] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6556ccbe398347e9a7d4c58486d6508f, State: Initialized, Role: FOLLOWER
I20251210 21:03:50.551802 23564 consensus_queue.cc:260] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556ccbe398347e9a7d4c58486d6508f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 36625 } }
I20251210 21:03:50.552079 23564 raft_consensus.cc:399] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:50.552214 23564 raft_consensus.cc:493] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:50.552342 23564 raft_consensus.cc:3060] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:50.554953 23564 raft_consensus.cc:515] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556ccbe398347e9a7d4c58486d6508f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 36625 } }
I20251210 21:03:50.555279 23564 leader_election.cc:304] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6556ccbe398347e9a7d4c58486d6508f; no voters:
I20251210 21:03:50.562801 23576 raft_consensus.cc:2804] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:50.563100 23564 leader_election.cc:290] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:50.563362 23576 raft_consensus.cc:697] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 1 LEADER]: Becoming Leader. State: Replica: 6556ccbe398347e9a7d4c58486d6508f, State: Running, Role: LEADER
I20251210 21:03:50.563855 23576 consensus_queue.cc:237] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556ccbe398347e9a7d4c58486d6508f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 36625 } }
I20251210 21:03:50.564028 23564 ts_tablet_manager.cc:1434] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20251210 21:03:50.565891 23075 catalog_manager.cc:5654] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6556ccbe398347e9a7d4c58486d6508f (127.21.127.66). New cstate: current_term: 1 leader_uuid: "6556ccbe398347e9a7d4c58486d6508f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6556ccbe398347e9a7d4c58486d6508f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 36625 } health_report { overall_health: HEALTHY } } }
I20251210 21:03:50.572686 22013 master.cc:561] Master@127.21.127.126:37635 shutting down...
I20251210 21:03:50.579001 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:50.579340 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:50.579497 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P cd0782e39dd641eb8551779a94411c1a: stopping tablet replica
I20251210 21:03:50.594308 22013 master.cc:583] Master@127.21.127.126:37635 shutdown complete.
W20251210 21:03:51.551200 23513 heartbeater.cc:646] Failed to heartbeat to 127.21.127.126:37635 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111)
I20251210 21:03:52.071861 23584 raft_consensus.cc:493] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd0782e39dd641eb8551779a94411c1a)
I20251210 21:03:52.071938 23585 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd0782e39dd641eb8551779a94411c1a)
I20251210 21:03:52.072075 23584 raft_consensus.cc:515] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:52.072096 23585 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:52.072780 23584 leader_election.cc:290] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635), 1cfd9bdacfd04f809ff6e5d686af636c (127.21.127.125:33435)
I20251210 21:03:52.073024 23585 leader_election.cc:290] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635), a1d81542db044c5da6ff2167351201fe (127.21.127.124:35919)
I20251210 21:03:52.073561 23216 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "a1d81542db044c5da6ff2167351201fe" is_pre_election: true
I20251210 21:03:52.073824 23216 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cfd9bdacfd04f809ff6e5d686af636c in term 1.
I20251210 21:03:52.073820 23151 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1d81542db044c5da6ff2167351201fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" is_pre_election: true
I20251210 21:03:52.074218 23151 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d81542db044c5da6ff2167351201fe in term 1.
I20251210 21:03:52.074458 23124 leader_election.cc:304] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cfd9bdacfd04f809ff6e5d686af636c, a1d81542db044c5da6ff2167351201fe; no voters:
I20251210 21:03:52.074757 23585 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:03:52.074884 23585 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd0782e39dd641eb8551779a94411c1a)
I20251210 21:03:52.074999 23585 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:03:52.075079 23189 leader_election.cc:304] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cfd9bdacfd04f809ff6e5d686af636c, a1d81542db044c5da6ff2167351201fe; no voters:
I20251210 21:03:52.075322 23584 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:03:52.075431 23584 raft_consensus.cc:493] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd0782e39dd641eb8551779a94411c1a)
I20251210 21:03:52.075568 23584 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 1 FOLLOWER]: Advancing to term 2
W20251210 21:03:52.076140 23125 leader_election.cc:336] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635): Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111)
W20251210 21:03:52.076504 23190 leader_election.cc:336] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635): Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111)
I20251210 21:03:52.077873 23585 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:52.078002 23584 raft_consensus.cc:515] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:52.078502 23585 leader_election.cc:290] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [CANDIDATE]: Term 2 election: Requested vote from peers cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635), a1d81542db044c5da6ff2167351201fe (127.21.127.124:35919)
I20251210 21:03:52.078500 23584 leader_election.cc:290] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 2 election: Requested vote from peers cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635), 1cfd9bdacfd04f809ff6e5d686af636c (127.21.127.125:33435)
I20251210 21:03:52.079013 23216 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "a1d81542db044c5da6ff2167351201fe"
I20251210 21:03:52.079304 23216 raft_consensus.cc:2393] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1cfd9bdacfd04f809ff6e5d686af636c in current term 2: Already voted for candidate a1d81542db044c5da6ff2167351201fe in this term.
I20251210 21:03:52.079432 23151 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1d81542db044c5da6ff2167351201fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "1cfd9bdacfd04f809ff6e5d686af636c"
I20251210 21:03:52.079668 23151 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a1d81542db044c5da6ff2167351201fe in current term 2: Already voted for candidate 1cfd9bdacfd04f809ff6e5d686af636c in this term.
W20251210 21:03:52.079799 23125 leader_election.cc:336] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635): Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111)
I20251210 21:03:52.079986 23124 leader_election.cc:304] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cfd9bdacfd04f809ff6e5d686af636c; no voters: a1d81542db044c5da6ff2167351201fe, cd0782e39dd641eb8551779a94411c1a
I20251210 21:03:52.080260 23585 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20251210 21:03:52.080406 23190 leader_election.cc:336] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635): Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111)
I20251210 21:03:52.080523 23190 leader_election.cc:304] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1d81542db044c5da6ff2167351201fe; no voters: 1cfd9bdacfd04f809ff6e5d686af636c, cd0782e39dd641eb8551779a94411c1a
I20251210 21:03:52.080757 23584 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251210 21:03:53.863500 23596 raft_consensus.cc:493] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:53.863698 23596 raft_consensus.cc:515] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:53.864509 23596 leader_election.cc:290] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635), 1cfd9bdacfd04f809ff6e5d686af636c (127.21.127.125:33435)
I20251210 21:03:53.865103 23151 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1d81542db044c5da6ff2167351201fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" is_pre_election: true
I20251210 21:03:53.865293 23151 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d81542db044c5da6ff2167351201fe in term 2.
W20251210 21:03:53.865391 23190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111) [suppressed 11 similar messages]
I20251210 21:03:53.865809 23189 leader_election.cc:304] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cfd9bdacfd04f809ff6e5d686af636c, a1d81542db044c5da6ff2167351201fe; no voters:
I20251210 21:03:53.866140 23596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251210 21:03:53.866264 23596 raft_consensus.cc:493] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:53.866381 23596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 2 FOLLOWER]: Advancing to term 3
W20251210 21:03:53.866899 23190 leader_election.cc:336] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635): Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111)
I20251210 21:03:53.868276 23596 raft_consensus.cc:515] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:53.868852 23596 leader_election.cc:290] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 3 election: Requested vote from peers cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635), 1cfd9bdacfd04f809ff6e5d686af636c (127.21.127.125:33435)
I20251210 21:03:53.869407 23151 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1d81542db044c5da6ff2167351201fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "1cfd9bdacfd04f809ff6e5d686af636c"
I20251210 21:03:53.869583 23151 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 2 FOLLOWER]: Advancing to term 3
W20251210 21:03:53.870055 23190 leader_election.cc:336] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635): Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111)
I20251210 21:03:53.871570 23151 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d81542db044c5da6ff2167351201fe in term 3.
I20251210 21:03:53.871974 23189 leader_election.cc:304] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cfd9bdacfd04f809ff6e5d686af636c, a1d81542db044c5da6ff2167351201fe; no voters: cd0782e39dd641eb8551779a94411c1a
I20251210 21:03:53.872257 23596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:03:53.872634 23596 raft_consensus.cc:697] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 3 LEADER]: Becoming Leader. State: Replica: a1d81542db044c5da6ff2167351201fe, State: Running, Role: LEADER
I20251210 21:03:53.872886 23596 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } }
I20251210 21:03:53.874114 23599 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1d81542db044c5da6ff2167351201fe. Latest consensus state: current_term: 3 leader_uuid: "a1d81542db044c5da6ff2167351201fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0782e39dd641eb8551779a94411c1a" member_type: VOTER last_known_addr { host: "127.21.127.126" port: 37635 } } peers { permanent_uuid: "1cfd9bdacfd04f809ff6e5d686af636c" member_type: VOTER last_known_addr { host: "127.21.127.125" port: 33435 } } peers { permanent_uuid: "a1d81542db044c5da6ff2167351201fe" member_type: VOTER last_known_addr { host: "127.21.127.124" port: 35919 } } }
I20251210 21:03:53.874395 23599 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:53.874856 23601 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:03:53.876546 23601 catalog_manager.cc:679] Loaded metadata for table test-workload [id=2b057417f5184bbeaae9acc10d9d7345]
I20251210 21:03:53.877862 23601 tablet_loader.cc:96] loaded metadata for tablet 0422b8cfc21f45c5a96bf18b873bf9a8 (table test-workload [id=2b057417f5184bbeaae9acc10d9d7345])
I20251210 21:03:53.878242 23601 tablet_loader.cc:96] loaded metadata for tablet 390e5d5b6ee747c0a0621217f2f07c54 (table test-workload [id=2b057417f5184bbeaae9acc10d9d7345])
I20251210 21:03:53.878569 23601 tablet_loader.cc:96] loaded metadata for tablet b2536231db684e538d76e8638bb12647 (table test-workload [id=2b057417f5184bbeaae9acc10d9d7345])
I20251210 21:03:53.878789 23601 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:03:53.879660 23601 catalog_manager.cc:1269] Loaded cluster ID: b8851b687f4840fd806e6711cdc4ddfe
I20251210 21:03:53.879740 23601 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:03:53.880846 23601 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:03:53.881649 23601 catalog_manager.cc:6038] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: Loaded TSK: 0
I20251210 21:03:53.882258 23601 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:03:54.055480 23282 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:03:54.055622 23282 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:03:54.174237 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:03:54.184844 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:54.185159 22013 tablet_replica.cc:333] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089: stopping tablet replica
I20251210 21:03:54.185345 22013 raft_consensus.cc:2243] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:54.185542 22013 raft_consensus.cc:2272] T 0422b8cfc21f45c5a96bf18b873bf9a8 P f6432d4a30ec469c99dbbdedd5aee089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:54.199095 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:03:54.201975 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:03:54.209913 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:54.210242 22013 tablet_replica.cc:333] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f: stopping tablet replica
I20251210 21:03:54.210431 22013 raft_consensus.cc:2243] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:54.210603 22013 raft_consensus.cc:2272] T b2536231db684e538d76e8638bb12647 P 6556ccbe398347e9a7d4c58486d6508f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:54.224059 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:03:54.226928 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:03:54.235263 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:03:54.235582 22013 tablet_replica.cc:333] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f: stopping tablet replica
I20251210 21:03:54.235811 22013 raft_consensus.cc:2243] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:03:54.235999 22013 raft_consensus.cc:2272] T 390e5d5b6ee747c0a0621217f2f07c54 P 250678d1c696453ca9eb750d0325360f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:54.249624 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:03:54.252487 22013 master.cc:561] Master@127.21.127.125:33435 shutting down...
I20251210 21:03:54.259549 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:03:54.259814 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:54.259974 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1cfd9bdacfd04f809ff6e5d686af636c: stopping tablet replica
W20251210 21:03:54.261235 23189 consensus_peers.cc:597] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe -> Peer 1cfd9bdacfd04f809ff6e5d686af636c (127.21.127.125:33435): Couldn't send request to peer 1cfd9bdacfd04f809ff6e5d686af636c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:03:54.273738 22013 master.cc:583] Master@127.21.127.125:33435 shutdown complete.
W20251210 21:03:54.275148 23190 consensus_peers.cc:597] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe -> Peer cd0782e39dd641eb8551779a94411c1a (127.21.127.126:37635): Couldn't send request to peer cd0782e39dd641eb8551779a94411c1a. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.126:37635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:03:54.277536 22013 master.cc:561] Master@127.21.127.124:35919 shutting down...
I20251210 21:03:54.284792 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 3 LEADER]: Raft consensus shutting down.
I20251210 21:03:54.285143 22013 pending_rounds.cc:70] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: Trying to abort 1 pending ops.
I20251210 21:03:54.285323 22013 pending_rounds.cc:77] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: Aborting op as it isn't in flight: id { term: 3 index: 11 } timestamp: 7231080996345348096 op_type: NO_OP noop_request { }
I20251210 21:03:54.285486 22013 raft_consensus.cc:2889] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:03:54.285612 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:03:54.285815 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P a1d81542db044c5da6ff2167351201fe: stopping tablet replica
I20251210 21:03:54.299556 22013 master.cc:583] Master@127.21.127.124:35919 shutdown complete.
[ OK ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing (6423 ms)
[ RUN ] AutoRebalancerTest.MovesScheduledIfAddTserver
I20251210 21:03:54.311371 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:34685
I20251210 21:03:54.311950 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:54.314525 23602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.314536 23603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.314723 23605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:54.315182 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:54.315464 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:54.315549 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:54.315603 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400634315586 us; error 0 us; skew 500 ppm
I20251210 21:03:54.315799 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:54.317011 22013 webserver.cc:492] Webserver started at http://127.21.127.126:43237/ using document root <none> and password file <none>
I20251210 21:03:54.317247 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:54.317320 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:54.317574 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:54.318320 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "ec353632f02543fdb9746d42d3f2379c"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.320873 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:03:54.322458 23610 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.322816 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:54.322921 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "ec353632f02543fdb9746d42d3f2379c"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.323041 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:54.343921 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:54.344584 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:54.357668 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:34685
I20251210 21:03:54.357692 23661 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:34685 every 8 connection(s)
I20251210 21:03:54.359172 23662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:54.363224 23662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c: Bootstrap starting.
I20251210 21:03:54.364650 23662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:54.366360 23662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c: No bootstrap required, opened a new log
I20251210 21:03:54.367039 23662 raft_consensus.cc:359] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec353632f02543fdb9746d42d3f2379c" member_type: VOTER }
I20251210 21:03:54.367174 23662 raft_consensus.cc:385] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:54.367266 23662 raft_consensus.cc:740] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec353632f02543fdb9746d42d3f2379c, State: Initialized, Role: FOLLOWER
I20251210 21:03:54.367504 23662 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec353632f02543fdb9746d42d3f2379c" member_type: VOTER }
I20251210 21:03:54.367646 23662 raft_consensus.cc:399] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:03:54.367738 23662 raft_consensus.cc:493] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:03:54.367842 23662 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:54.369585 23662 raft_consensus.cc:515] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec353632f02543fdb9746d42d3f2379c" member_type: VOTER }
I20251210 21:03:54.369876 23662 leader_election.cc:304] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec353632f02543fdb9746d42d3f2379c; no voters:
I20251210 21:03:54.370297 23662 leader_election.cc:290] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:03:54.370427 23665 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:54.370909 23665 raft_consensus.cc:697] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 1 LEADER]: Becoming Leader. State: Replica: ec353632f02543fdb9746d42d3f2379c, State: Running, Role: LEADER
I20251210 21:03:54.371191 23662 sys_catalog.cc:565] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:03:54.371165 23665 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec353632f02543fdb9746d42d3f2379c" member_type: VOTER }
I20251210 21:03:54.372023 23666 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec353632f02543fdb9746d42d3f2379c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec353632f02543fdb9746d42d3f2379c" member_type: VOTER } }
I20251210 21:03:54.372097 23667 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec353632f02543fdb9746d42d3f2379c. Latest consensus state: current_term: 1 leader_uuid: "ec353632f02543fdb9746d42d3f2379c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec353632f02543fdb9746d42d3f2379c" member_type: VOTER } }
I20251210 21:03:54.372339 23666 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:54.372346 23667 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [sys.catalog]: This master's current role is: LEADER
I20251210 21:03:54.373291 23671 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:03:54.374716 23671 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:03:54.377846 23671 catalog_manager.cc:1357] Generated new cluster ID: f9b5a7c7ca444cd0819a23205b1e3eef
I20251210 21:03:54.377969 23671 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:03:54.378912 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:03:54.394320 23671 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:03:54.395113 23671 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:03:54.404569 23671 catalog_manager.cc:6027] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c: Generated new TSK 0
I20251210 21:03:54.404824 23671 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:03:54.411696 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:54.414390 23683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.415076 23684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.415736 23686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:54.415824 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:54.416136 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:54.416198 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:54.416253 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400634416229 us; error 0 us; skew 500 ppm
I20251210 21:03:54.416378 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:54.417541 22013 webserver.cc:492] Webserver started at http://127.21.127.65:40043/ using document root <none> and password file <none>
I20251210 21:03:54.417748 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:54.417825 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:54.417941 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:54.418560 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "1615c84f9a134ec58ba4c972903ae15d"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.421046 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:03:54.422612 23691 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.422972 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:03:54.423074 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "1615c84f9a134ec58ba4c972903ae15d"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.423207 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:54.431881 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:54.432451 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:54.432986 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:54.433861 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:54.433930 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.434051 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:54.434108 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.448247 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:44433
I20251210 21:03:54.448294 23753 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:44433 every 8 connection(s)
I20251210 21:03:54.449858 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:54.453069 23758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.453297 23759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.454144 23761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:54.455516 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:54.455875 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:54.455986 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:54.456048 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400634456026 us; error 0 us; skew 500 ppm
I20251210 21:03:54.456256 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:54.457257 22013 webserver.cc:492] Webserver started at http://127.21.127.66:39853/ using document root <none> and password file <none>
I20251210 21:03:54.457481 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:54.457557 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:54.457693 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:54.458264 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "2ce37c31a32541bd8b48d9ffca9f47e9"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.460922 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:03:54.461149 23754 heartbeater.cc:344] Connected to a master server at 127.21.127.126:34685
I20251210 21:03:54.461292 23754 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:54.461702 23754 heartbeater.cc:507] Master 127.21.127.126:34685 requested a full tablet report, sending...
I20251210 21:03:54.462641 23627 ts_manager.cc:194] Registered new tserver with Master: 1615c84f9a134ec58ba4c972903ae15d (127.21.127.65:44433)
I20251210 21:03:54.462836 23766 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.463179 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:03:54.463295 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "2ce37c31a32541bd8b48d9ffca9f47e9"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.463419 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:54.463989 23627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38168
I20251210 21:03:54.510902 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:54.511422 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:54.512014 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:54.512802 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:54.512859 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.512924 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:54.512957 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.524749 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:43527
I20251210 21:03:54.524767 23828 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:43527 every 8 connection(s)
I20251210 21:03:54.526185 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:03:54.528721 23833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.528743 23832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:03:54.529137 23835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:03:54.529819 22013 server_base.cc:1047] running on GCE node
I20251210 21:03:54.530092 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:03:54.530158 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:03:54.530238 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400634530233 us; error 0 us; skew 500 ppm
I20251210 21:03:54.530395 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:03:54.531675 22013 webserver.cc:492] Webserver started at http://127.21.127.67:36815/ using document root <none> and password file <none>
I20251210 21:03:54.531896 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:03:54.531956 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:03:54.532066 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:03:54.532198 23829 heartbeater.cc:344] Connected to a master server at 127.21.127.126:34685
I20251210 21:03:54.532318 23829 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:54.532634 23829 heartbeater.cc:507] Master 127.21.127.126:34685 requested a full tablet report, sending...
I20251210 21:03:54.532748 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "81e7531643c0403ca29845d0978222fb"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.533455 23627 ts_manager.cc:194] Registered new tserver with Master: 2ce37c31a32541bd8b48d9ffca9f47e9 (127.21.127.66:43527)
I20251210 21:03:54.534196 23627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38170
I20251210 21:03:54.535027 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:03:54.536454 23840 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.536791 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:03:54.536871 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "81e7531643c0403ca29845d0978222fb"
format_stamp: "Formatted at 2025-12-10 21:03:54 on dist-test-slave-9pq8"
I20251210 21:03:54.536968 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:03:54.553045 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:03:54.553898 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:03:54.554513 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:03:54.555290 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:03:54.555344 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.555398 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:03:54.555426 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:03:54.567387 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:39539
I20251210 21:03:54.567417 23902 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:39539 every 8 connection(s)
I20251210 21:03:54.573768 23903 heartbeater.cc:344] Connected to a master server at 127.21.127.126:34685
I20251210 21:03:54.573921 23903 heartbeater.cc:461] Registering TS with master...
I20251210 21:03:54.574258 23903 heartbeater.cc:507] Master 127.21.127.126:34685 requested a full tablet report, sending...
I20251210 21:03:54.575093 23627 ts_manager.cc:194] Registered new tserver with Master: 81e7531643c0403ca29845d0978222fb (127.21.127.67:39539)
I20251210 21:03:54.575858 23627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38186
I20251210 21:03:54.576129 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007918394s
I20251210 21:03:55.465494 23754 heartbeater.cc:499] Master 127.21.127.126:34685 was elected leader, sending a full tablet report...
I20251210 21:03:55.535398 23829 heartbeater.cc:499] Master 127.21.127.126:34685 was elected leader, sending a full tablet report...
I20251210 21:03:55.577066 23903 heartbeater.cc:499] Master 127.21.127.126:34685 was elected leader, sending a full tablet report...
I20251210 21:03:55.604475 22013 test_util.cc:276] Using random seed: 1573223894
I20251210 21:03:55.617514 23627 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38192:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:03:55.618189 23627 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:03:55.638573 23794 tablet_service.cc:1505] Processing CreateTablet for tablet 3f5b6ae2675e430e887d81cc0b3a45e1 (DEFAULT_TABLE table=test-workload [id=c0178f8caa1d4cf5878b471831f9c3f5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251210 21:03:55.638574 23867 tablet_service.cc:1505] Processing CreateTablet for tablet e97a253140f744a995d80374eb769693 (DEFAULT_TABLE table=test-workload [id=c0178f8caa1d4cf5878b471831f9c3f5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251210 21:03:55.639098 23867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97a253140f744a995d80374eb769693. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:55.639118 23794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f5b6ae2675e430e887d81cc0b3a45e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:55.638574 23868 tablet_service.cc:1505] Processing CreateTablet for tablet 3f5b6ae2675e430e887d81cc0b3a45e1 (DEFAULT_TABLE table=test-workload [id=c0178f8caa1d4cf5878b471831f9c3f5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251210 21:03:55.639864 23868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f5b6ae2675e430e887d81cc0b3a45e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:55.643075 23718 tablet_service.cc:1505] Processing CreateTablet for tablet e97a253140f744a995d80374eb769693 (DEFAULT_TABLE table=test-workload [id=c0178f8caa1d4cf5878b471831f9c3f5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251210 21:03:55.643409 23718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97a253140f744a995d80374eb769693. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:55.645618 23923 tablet_bootstrap.cc:492] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb: Bootstrap starting.
I20251210 21:03:55.647399 23923 tablet_bootstrap.cc:654] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:55.643075 23719 tablet_service.cc:1505] Processing CreateTablet for tablet 3f5b6ae2675e430e887d81cc0b3a45e1 (DEFAULT_TABLE table=test-workload [id=c0178f8caa1d4cf5878b471831f9c3f5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251210 21:03:55.647922 23719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f5b6ae2675e430e887d81cc0b3a45e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:55.649250 23923 tablet_bootstrap.cc:492] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb: No bootstrap required, opened a new log
I20251210 21:03:55.649395 23923 ts_tablet_manager.cc:1403] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251210 21:03:55.650079 23923 raft_consensus.cc:359] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:55.650249 23923 raft_consensus.cc:385] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:55.650300 23923 raft_consensus.cc:740] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e7531643c0403ca29845d0978222fb, State: Initialized, Role: FOLLOWER
I20251210 21:03:55.650492 23923 consensus_queue.cc:260] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:55.651197 23923 ts_tablet_manager.cc:1434] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:03:55.651496 23923 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: Bootstrap starting.
I20251210 21:03:55.653067 23923 tablet_bootstrap.cc:654] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:55.655165 23793 tablet_service.cc:1505] Processing CreateTablet for tablet e97a253140f744a995d80374eb769693 (DEFAULT_TABLE table=test-workload [id=c0178f8caa1d4cf5878b471831f9c3f5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251210 21:03:55.655638 23793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97a253140f744a995d80374eb769693. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:03:55.659197 23928 tablet_bootstrap.cc:492] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d: Bootstrap starting.
I20251210 21:03:55.660897 23928 tablet_bootstrap.cc:654] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:55.662437 23926 tablet_bootstrap.cc:492] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9: Bootstrap starting.
I20251210 21:03:55.663972 23926 tablet_bootstrap.cc:654] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:55.664295 23928 tablet_bootstrap.cc:492] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d: No bootstrap required, opened a new log
I20251210 21:03:55.664511 23928 ts_tablet_manager.cc:1403] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:03:55.664530 23923 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: No bootstrap required, opened a new log
I20251210 21:03:55.664639 23923 ts_tablet_manager.cc:1403] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20251210 21:03:55.665201 23928 raft_consensus.cc:359] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:55.665371 23928 raft_consensus.cc:385] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:55.665279 23923 raft_consensus.cc:359] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.665467 23928 raft_consensus.cc:740] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1615c84f9a134ec58ba4c972903ae15d, State: Initialized, Role: FOLLOWER
I20251210 21:03:55.665513 23923 raft_consensus.cc:385] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:55.665585 23923 raft_consensus.cc:740] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e7531643c0403ca29845d0978222fb, State: Initialized, Role: FOLLOWER
I20251210 21:03:55.665738 23926 tablet_bootstrap.cc:492] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9: No bootstrap required, opened a new log
I20251210 21:03:55.665733 23928 consensus_queue.cc:260] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:55.665769 23923 consensus_queue.cc:260] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.665910 23926 ts_tablet_manager.cc:1403] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:03:55.666278 23923 ts_tablet_manager.cc:1434] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:03:55.666528 23926 raft_consensus.cc:359] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:55.666664 23926 raft_consensus.cc:385] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:55.666764 23926 raft_consensus.cc:740] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce37c31a32541bd8b48d9ffca9f47e9, State: Initialized, Role: FOLLOWER
I20251210 21:03:55.666976 23926 consensus_queue.cc:260] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:55.668049 23926 ts_tablet_manager.cc:1434] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:03:55.668329 23926 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9: Bootstrap starting.
I20251210 21:03:55.669859 23926 tablet_bootstrap.cc:654] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:55.671618 23928 ts_tablet_manager.cc:1434] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20251210 21:03:55.671873 23928 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: Bootstrap starting.
I20251210 21:03:55.672222 23926 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9: No bootstrap required, opened a new log
I20251210 21:03:55.672382 23926 ts_tablet_manager.cc:1403] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:03:55.672920 23926 raft_consensus.cc:359] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.673045 23926 raft_consensus.cc:385] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:55.673115 23926 raft_consensus.cc:740] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce37c31a32541bd8b48d9ffca9f47e9, State: Initialized, Role: FOLLOWER
I20251210 21:03:55.673484 23928 tablet_bootstrap.cc:654] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: Neither blocks nor log segments found. Creating new log.
I20251210 21:03:55.673795 23926 consensus_queue.cc:260] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.674157 23926 ts_tablet_manager.cc:1434] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:03:55.675280 23928 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: No bootstrap required, opened a new log
I20251210 21:03:55.675387 23928 ts_tablet_manager.cc:1403] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:03:55.675982 23928 raft_consensus.cc:359] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.676105 23928 raft_consensus.cc:385] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:03:55.676157 23928 raft_consensus.cc:740] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1615c84f9a134ec58ba4c972903ae15d, State: Initialized, Role: FOLLOWER
I20251210 21:03:55.676340 23928 consensus_queue.cc:260] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.676776 23928 ts_tablet_manager.cc:1434] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:03:55.786986 23932 raft_consensus.cc:493] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:55.787196 23932 raft_consensus.cc:515] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.788185 23932 leader_election.cc:290] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1615c84f9a134ec58ba4c972903ae15d (127.21.127.65:44433), 81e7531643c0403ca29845d0978222fb (127.21.127.67:39539)
I20251210 21:03:55.795187 23729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e97a253140f744a995d80374eb769693" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1615c84f9a134ec58ba4c972903ae15d" is_pre_election: true
I20251210 21:03:55.795475 23729 raft_consensus.cc:2468] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 0.
I20251210 21:03:55.795570 23878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e97a253140f744a995d80374eb769693" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e7531643c0403ca29845d0978222fb" is_pre_election: true
I20251210 21:03:55.795782 23878 raft_consensus.cc:2468] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 0.
I20251210 21:03:55.795949 23770 leader_election.cc:304] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1615c84f9a134ec58ba4c972903ae15d, 2ce37c31a32541bd8b48d9ffca9f47e9; no voters:
I20251210 21:03:55.796193 23932 raft_consensus.cc:2804] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:55.796284 23932 raft_consensus.cc:493] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:55.796356 23932 raft_consensus.cc:3060] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:55.798192 23932 raft_consensus.cc:515] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.798736 23932 leader_election.cc:290] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 election: Requested vote from peers 1615c84f9a134ec58ba4c972903ae15d (127.21.127.65:44433), 81e7531643c0403ca29845d0978222fb (127.21.127.67:39539)
I20251210 21:03:55.799165 23878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e97a253140f744a995d80374eb769693" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e7531643c0403ca29845d0978222fb"
I20251210 21:03:55.799165 23729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e97a253140f744a995d80374eb769693" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1615c84f9a134ec58ba4c972903ae15d"
I20251210 21:03:55.799353 23729 raft_consensus.cc:3060] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:55.799353 23878 raft_consensus.cc:3060] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:55.801964 23878 raft_consensus.cc:2468] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 1.
I20251210 21:03:55.801964 23729 raft_consensus.cc:2468] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 1.
I20251210 21:03:55.802353 23770 leader_election.cc:304] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1615c84f9a134ec58ba4c972903ae15d, 2ce37c31a32541bd8b48d9ffca9f47e9; no voters:
I20251210 21:03:55.802588 23932 raft_consensus.cc:2804] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:55.803017 23932 raft_consensus.cc:697] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 LEADER]: Becoming Leader. State: Replica: 2ce37c31a32541bd8b48d9ffca9f47e9, State: Running, Role: LEADER
I20251210 21:03:55.803236 23932 consensus_queue.cc:237] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:55.805522 23626 catalog_manager.cc:5654] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ce37c31a32541bd8b48d9ffca9f47e9 (127.21.127.66). New cstate: current_term: 1 leader_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:55.996451 23932 raft_consensus.cc:493] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:03:55.996646 23932 raft_consensus.cc:515] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:55.997295 23932 leader_election.cc:290] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81e7531643c0403ca29845d0978222fb (127.21.127.67:39539), 1615c84f9a134ec58ba4c972903ae15d (127.21.127.65:44433)
I20251210 21:03:55.997826 23878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f5b6ae2675e430e887d81cc0b3a45e1" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e7531643c0403ca29845d0978222fb" is_pre_election: true
I20251210 21:03:55.997831 23729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f5b6ae2675e430e887d81cc0b3a45e1" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1615c84f9a134ec58ba4c972903ae15d" is_pre_election: true
I20251210 21:03:55.998010 23878 raft_consensus.cc:2468] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 0.
I20251210 21:03:55.998059 23729 raft_consensus.cc:2468] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 0.
I20251210 21:03:55.998347 23767 leader_election.cc:304] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce37c31a32541bd8b48d9ffca9f47e9, 81e7531643c0403ca29845d0978222fb; no voters:
I20251210 21:03:55.998589 23932 raft_consensus.cc:2804] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:03:55.998675 23932 raft_consensus.cc:493] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:03:55.998720 23932 raft_consensus.cc:3060] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:56.000535 23932 raft_consensus.cc:515] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:56.001046 23932 leader_election.cc:290] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 election: Requested vote from peers 81e7531643c0403ca29845d0978222fb (127.21.127.67:39539), 1615c84f9a134ec58ba4c972903ae15d (127.21.127.65:44433)
I20251210 21:03:56.001461 23729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f5b6ae2675e430e887d81cc0b3a45e1" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1615c84f9a134ec58ba4c972903ae15d"
I20251210 21:03:56.001446 23878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f5b6ae2675e430e887d81cc0b3a45e1" candidate_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e7531643c0403ca29845d0978222fb"
I20251210 21:03:56.001629 23878 raft_consensus.cc:3060] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:56.001683 23729 raft_consensus.cc:3060] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:03:56.004251 23878 raft_consensus.cc:2468] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 1.
I20251210 21:03:56.004271 23729 raft_consensus.cc:2468] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce37c31a32541bd8b48d9ffca9f47e9 in term 1.
I20251210 21:03:56.004642 23767 leader_election.cc:304] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce37c31a32541bd8b48d9ffca9f47e9, 81e7531643c0403ca29845d0978222fb; no voters:
I20251210 21:03:56.004956 23932 raft_consensus.cc:2804] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:03:56.005156 23932 raft_consensus.cc:697] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 LEADER]: Becoming Leader. State: Replica: 2ce37c31a32541bd8b48d9ffca9f47e9, State: Running, Role: LEADER
I20251210 21:03:56.005508 23932 consensus_queue.cc:237] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } }
I20251210 21:03:56.008185 23626 catalog_manager.cc:5654] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ce37c31a32541bd8b48d9ffca9f47e9 (127.21.127.66). New cstate: current_term: 1 leader_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:56.228448 23932 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:56.232973 23932 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:56.378872 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:03:56.385773 23804 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e97a253140f744a995d80374eb769693"
dest_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9"
mode: GRACEFUL
new_leader_uuid: "1615c84f9a134ec58ba4c972903ae15d"
from {username='slave'} at 127.0.0.1:57452
I20251210 21:03:56.386020 23804 raft_consensus.cc:606] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 LEADER]: Received request to transfer leadership to TS 1615c84f9a134ec58ba4c972903ae15d
I20251210 21:03:56.386538 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:03:56.386693 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:03:56.577291 23932 consensus_queue.cc:1048] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:56.579195 23932 consensus_queue.cc:1048] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:03:56.628468 23937 raft_consensus.cc:993] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9: : Instructing follower 1615c84f9a134ec58ba4c972903ae15d to start an election
I20251210 21:03:56.628655 23937 raft_consensus.cc:1081] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 LEADER]: Signalling peer 1615c84f9a134ec58ba4c972903ae15d to start an election
I20251210 21:03:56.629249 23729 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e97a253140f744a995d80374eb769693"
dest_uuid: "1615c84f9a134ec58ba4c972903ae15d"
from {username='slave'} at 127.0.0.1:41612
I20251210 21:03:56.629429 23729 raft_consensus.cc:493] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:03:56.629563 23729 raft_consensus.cc:3060] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:03:56.631310 23729 raft_consensus.cc:515] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:56.632030 23729 leader_election.cc:290] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [CANDIDATE]: Term 2 election: Requested vote from peers 2ce37c31a32541bd8b48d9ffca9f47e9 (127.21.127.66:43527), 81e7531643c0403ca29845d0978222fb (127.21.127.67:39539)
I20251210 21:03:56.638219 23878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e97a253140f744a995d80374eb769693" candidate_uuid: "1615c84f9a134ec58ba4c972903ae15d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "81e7531643c0403ca29845d0978222fb"
I20251210 21:03:56.638329 23804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e97a253140f744a995d80374eb769693" candidate_uuid: "1615c84f9a134ec58ba4c972903ae15d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9"
I20251210 21:03:56.638466 23878 raft_consensus.cc:3060] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:03:56.638545 23804 raft_consensus.cc:3055] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:03:56.638701 23804 raft_consensus.cc:740] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ce37c31a32541bd8b48d9ffca9f47e9, State: Running, Role: LEADER
I20251210 21:03:56.638985 23804 consensus_queue.cc:260] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:56.639304 23804 raft_consensus.cc:3060] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:03:56.640607 23878 raft_consensus.cc:2468] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1615c84f9a134ec58ba4c972903ae15d in term 2.
I20251210 21:03:56.641052 23692 leader_election.cc:304] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1615c84f9a134ec58ba4c972903ae15d, 81e7531643c0403ca29845d0978222fb; no voters:
I20251210 21:03:56.641263 23804 raft_consensus.cc:2468] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1615c84f9a134ec58ba4c972903ae15d in term 2.
I20251210 21:03:56.641360 23944 raft_consensus.cc:2804] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:03:56.641863 23944 raft_consensus.cc:697] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 LEADER]: Becoming Leader. State: Replica: 1615c84f9a134ec58ba4c972903ae15d, State: Running, Role: LEADER
I20251210 21:03:56.642167 23944 consensus_queue.cc:237] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } }
I20251210 21:03:56.644281 23626 catalog_manager.cc:5654] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d reported cstate change: term changed from 1 to 2, leader changed from 2ce37c31a32541bd8b48d9ffca9f47e9 (127.21.127.66) to 1615c84f9a134ec58ba4c972903ae15d (127.21.127.65). New cstate: current_term: 2 leader_uuid: "1615c84f9a134ec58ba4c972903ae15d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } health_report { overall_health: UNKNOWN } } }
I20251210 21:03:57.172482 23878 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:03:57.173058 23959 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:57.176698 23804 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:03:57.177400 23959 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:03:58.387086 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:03:58.387799 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:03:58.387938 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:00.388329 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:00.388945 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:00.389101 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:02.389448 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:02.390098 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:02.390233 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:04.390561 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:04.391144 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:04.391295 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:06.391697 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:06.392320 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:06.392438 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:07.146067 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:07.148792 23975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:07.149055 23976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:07.149468 23978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:07.150554 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:07.151021 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:07.151104 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:07.151165 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400647151145 us; error 0 us; skew 500 ppm
I20251210 21:04:07.151333 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:07.152706 22013 webserver.cc:492] Webserver started at http://127.21.127.68:33255/ using document root <none> and password file <none>
I20251210 21:04:07.152951 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:07.153029 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:07.153195 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:07.153844 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "43b44c0a9b4c485fa4e40157272b08bc"
format_stamp: "Formatted at 2025-12-10 21:04:07 on dist-test-slave-9pq8"
I20251210 21:04:07.156333 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:04:07.157976 23983 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:07.158380 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:07.158493 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "43b44c0a9b4c485fa4e40157272b08bc"
format_stamp: "Formatted at 2025-12-10 21:04:07 on dist-test-slave-9pq8"
I20251210 21:04:07.158598 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:07.172037 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:07.172645 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:07.173198 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:07.174053 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:07.174135 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:07.174204 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:07.174263 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:07.188570 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:38501
I20251210 21:04:07.188625 24045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:38501 every 8 connection(s)
I20251210 21:04:07.198694 24046 heartbeater.cc:344] Connected to a master server at 127.21.127.126:34685
I20251210 21:04:07.198850 24046 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:07.199275 24046 heartbeater.cc:507] Master 127.21.127.126:34685 requested a full tablet report, sending...
I20251210 21:04:07.200060 23626 ts_manager.cc:194] Registered new tserver with Master: 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68:38501)
I20251210 21:04:07.200829 23626 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53284
I20251210 21:04:07.403910 23729 consensus_queue.cc:237] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: true } }
I20251210 21:04:07.407076 23804 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:04:07.407076 23878 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:04:07.407833 24053 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:04:07.408212 23974 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:04:07.410892 23974 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: true } } }
I20251210 21:04:07.412199 23804 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: true } } }
I20251210 21:04:07.412716 23878 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: true } } }
W20251210 21:04:07.416411 23692 consensus_peers.cc:597] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d -> Peer 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68:38501): Couldn't send request to peer 43b44c0a9b4c485fa4e40157272b08bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e97a253140f744a995d80374eb769693. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:04:07.417261 23626 catalog_manager.cc:5654] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d reported cstate change: config changed from index -1 to 3, NON_VOTER 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "1615c84f9a134ec58ba4c972903ae15d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:04:07.856357 24063 ts_tablet_manager.cc:933] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: Initiating tablet copy from peer 1615c84f9a134ec58ba4c972903ae15d (127.21.127.65:44433)
I20251210 21:04:07.857416 24063 tablet_copy_client.cc:323] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:44433
I20251210 21:04:07.863162 23739 tablet_copy_service.cc:140] P 1615c84f9a134ec58ba4c972903ae15d: Received BeginTabletCopySession request for tablet e97a253140f744a995d80374eb769693 from peer 43b44c0a9b4c485fa4e40157272b08bc ({username='slave'} at 127.0.0.1:47312)
I20251210 21:04:07.863324 23739 tablet_copy_service.cc:161] P 1615c84f9a134ec58ba4c972903ae15d: Beginning new tablet copy session on tablet e97a253140f744a995d80374eb769693 from peer 43b44c0a9b4c485fa4e40157272b08bc at {username='slave'} at 127.0.0.1:47312: session id = 43b44c0a9b4c485fa4e40157272b08bc-e97a253140f744a995d80374eb769693
I20251210 21:04:07.866366 23739 tablet_copy_source_session.cc:215] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:04:07.868137 24063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97a253140f744a995d80374eb769693. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:07.873757 24063 tablet_copy_client.cc:806] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: tablet copy: Starting download of 0 data blocks...
I20251210 21:04:07.874034 24063 tablet_copy_client.cc:670] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: tablet copy: Starting download of 1 WAL segments...
I20251210 21:04:07.876106 24063 tablet_copy_client.cc:538] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:04:07.879639 24063 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: Bootstrap starting.
I20251210 21:04:07.886919 24063 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:04:07.887264 24063 tablet_bootstrap.cc:492] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: Bootstrap complete.
I20251210 21:04:07.887521 24063 ts_tablet_manager.cc:1403] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20251210 21:04:07.888135 24063 raft_consensus.cc:359] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: true } }
I20251210 21:04:07.888296 24063 raft_consensus.cc:740] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 43b44c0a9b4c485fa4e40157272b08bc, State: Initialized, Role: LEARNER
I20251210 21:04:07.888479 24063 consensus_queue.cc:260] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: true } }
I20251210 21:04:07.889197 24063 ts_tablet_manager.cc:1434] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:04:07.889204 24046 heartbeater.cc:499] Master 127.21.127.126:34685 was elected leader, sending a full tablet report...
I20251210 21:04:07.889874 23739 tablet_copy_service.cc:342] P 1615c84f9a134ec58ba4c972903ae15d: Request end of tablet copy session 43b44c0a9b4c485fa4e40157272b08bc-e97a253140f744a995d80374eb769693 received from {username='slave'} at 127.0.0.1:47312
I20251210 21:04:07.890013 23739 tablet_copy_service.cc:434] P 1615c84f9a134ec58ba4c972903ae15d: ending tablet copy session 43b44c0a9b4c485fa4e40157272b08bc-e97a253140f744a995d80374eb769693 on tablet e97a253140f744a995d80374eb769693 with peer 43b44c0a9b4c485fa4e40157272b08bc
I20251210 21:04:08.257222 24021 raft_consensus.cc:1217] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251210 21:04:08.392791 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:08.393332 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:08.393442 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:08.836717 24068 raft_consensus.cc:1064] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: attempting to promote NON_VOTER 43b44c0a9b4c485fa4e40157272b08bc to VOTER
I20251210 21:04:08.837564 24068 consensus_queue.cc:237] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } }
I20251210 21:04:08.839673 24021 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 LEARNER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:04:08.839982 23878 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:04:08.840492 24070 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:04:08.841104 24070 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:04:08.851420 23804 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:04:08.852082 24070 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:04:08.855193 24068 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } } }
I20251210 21:04:08.858343 23804 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } } }
I20251210 21:04:08.858539 24021 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } } }
I20251210 21:04:08.858918 23878 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } } }
I20251210 21:04:08.860538 23627 catalog_manager.cc:5654] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d reported cstate change: config changed from index 3 to 4, 43b44c0a9b4c485fa4e40157272b08bc (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "1615c84f9a134ec58ba4c972903ae15d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81e7531643c0403ca29845d0978222fb" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 39539 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:04:08.881826 23728 consensus_queue.cc:237] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } }
I20251210 21:04:08.883724 24021 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251210 21:04:08.883751 23804 raft_consensus.cc:1275] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Refusing update from remote peer 1615c84f9a134ec58ba4c972903ae15d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251210 21:04:08.884294 24068 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:04:08.884552 24070 consensus_queue.cc:1048] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:04:08.886593 24072 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 81e7531643c0403ca29845d0978222fb (127.21.127.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } } }
I20251210 21:04:08.887190 24021 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 81e7531643c0403ca29845d0978222fb (127.21.127.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } } }
I20251210 21:04:08.887845 23804 raft_consensus.cc:2955] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 81e7531643c0403ca29845d0978222fb (127.21.127.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } } }
I20251210 21:04:08.890035 23614 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet e97a253140f744a995d80374eb769693 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:04:08.891502 23627 catalog_manager.cc:5654] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d reported cstate change: config changed from index 4 to 5, VOTER 81e7531643c0403ca29845d0978222fb (127.21.127.67) evicted. New cstate: current_term: 2 leader_uuid: "1615c84f9a134ec58ba4c972903ae15d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1615c84f9a134ec58ba4c972903ae15d" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ce37c31a32541bd8b48d9ffca9f47e9" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43b44c0a9b4c485fa4e40157272b08bc" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 38501 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:04:08.895123 23868 tablet_service.cc:1552] Processing DeleteTablet for tablet e97a253140f744a995d80374eb769693 with delete_type TABLET_DATA_TOMBSTONED (TS 81e7531643c0403ca29845d0978222fb not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54478
I20251210 21:04:08.895972 24082 tablet_replica.cc:333] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: stopping tablet replica
I20251210 21:04:08.896111 24082 raft_consensus.cc:2243] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:04:08.896291 24082 raft_consensus.cc:2272] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:08.896976 24082 ts_tablet_manager.cc:1916] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:04:08.901121 24082 ts_tablet_manager.cc:1929] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20251210 21:04:08.901237 24082 log.cc:1199] T e97a253140f744a995d80374eb769693 P 81e7531643c0403ca29845d0978222fb: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/e97a253140f744a995d80374eb769693
I20251210 21:04:08.901862 23611 catalog_manager.cc:4985] TS 81e7531643c0403ca29845d0978222fb (127.21.127.67:39539): tablet e97a253140f744a995d80374eb769693 (table test-workload [id=c0178f8caa1d4cf5878b471831f9c3f5]) successfully deleted
I20251210 21:04:10.393869 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:10.394507 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:10.394635 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:12.395071 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:12.395748 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:12.395869 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:14.396246 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:14.396886 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:14.397008 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:16.397449 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:16.398110 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:16.398236 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:18.398684 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:18.399307 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:18.399422 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:20.399914 23681 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:20.400543 23681 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:20.400664 23681 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:21.015519 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:04:21.023985 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:21.024384 22013 tablet_replica.cc:333] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d: stopping tablet replica
I20251210 21:04:21.024626 22013 raft_consensus.cc:2243] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:04:21.024995 22013 raft_consensus.cc:2272] T e97a253140f744a995d80374eb769693 P 1615c84f9a134ec58ba4c972903ae15d [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:21.025641 22013 tablet_replica.cc:333] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d: stopping tablet replica
I20251210 21:04:21.025810 22013 raft_consensus.cc:2243] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:04:21.025969 22013 raft_consensus.cc:2272] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 1615c84f9a134ec58ba4c972903ae15d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:21.042928 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:04:21.046216 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:04:21.054335 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:21.054610 22013 tablet_replica.cc:333] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9: stopping tablet replica
I20251210 21:04:21.054831 22013 raft_consensus.cc:2243] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:04:21.055136 22013 raft_consensus.cc:2272] T e97a253140f744a995d80374eb769693 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:21.055845 22013 tablet_replica.cc:333] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9: stopping tablet replica
I20251210 21:04:21.055985 22013 raft_consensus.cc:2243] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:04:21.056291 22013 raft_consensus.cc:2272] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 2ce37c31a32541bd8b48d9ffca9f47e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:21.070115 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:04:21.073376 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:04:21.081172 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:21.081506 22013 tablet_replica.cc:333] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb: stopping tablet replica
I20251210 21:04:21.081682 22013 raft_consensus.cc:2243] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:04:21.081878 22013 raft_consensus.cc:2272] T 3f5b6ae2675e430e887d81cc0b3a45e1 P 81e7531643c0403ca29845d0978222fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:21.095594 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:04:21.098698 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:04:21.106761 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:21.107030 22013 tablet_replica.cc:333] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc: stopping tablet replica
I20251210 21:04:21.107214 22013 raft_consensus.cc:2243] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:04:21.107524 22013 raft_consensus.cc:2272] T e97a253140f744a995d80374eb769693 P 43b44c0a9b4c485fa4e40157272b08bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:21.121308 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:04:21.124394 22013 master.cc:561] Master@127.21.127.126:34685 shutting down...
I20251210 21:04:21.140053 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:04:21.140323 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:21.140453 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P ec353632f02543fdb9746d42d3f2379c: stopping tablet replica
I20251210 21:04:21.153988 22013 master.cc:583] Master@127.21.127.126:34685 shutdown complete.
[ OK ] AutoRebalancerTest.MovesScheduledIfAddTserver (26853 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTservers
I20251210 21:04:21.164906 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:40343
I20251210 21:04:21.165446 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:21.167924 24084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:21.168009 24085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:21.168476 24087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:21.168781 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:21.169003 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:21.169050 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:21.169078 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400661169075 us; error 0 us; skew 500 ppm
I20251210 21:04:21.169190 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:21.170269 22013 webserver.cc:492] Webserver started at http://127.21.127.126:36965/ using document root <none> and password file <none>
I20251210 21:04:21.170455 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:21.170507 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:21.170598 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:21.171204 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "ef9294ebd760498689c63a64473c665c"
format_stamp: "Formatted at 2025-12-10 21:04:21 on dist-test-slave-9pq8"
I20251210 21:04:21.173666 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20251210 21:04:21.175086 24092 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:21.175397 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:21.175508 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "ef9294ebd760498689c63a64473c665c"
format_stamp: "Formatted at 2025-12-10 21:04:21 on dist-test-slave-9pq8"
I20251210 21:04:21.175661 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:21.192884 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:21.193493 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:21.203795 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:40343
I20251210 21:04:21.203815 24143 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:40343 every 8 connection(s)
I20251210 21:04:21.205091 24144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:21.209165 24144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c: Bootstrap starting.
I20251210 21:04:21.210306 24144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c: Neither blocks nor log segments found. Creating new log.
I20251210 21:04:21.211833 24144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c: No bootstrap required, opened a new log
I20251210 21:04:21.212425 24144 raft_consensus.cc:359] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9294ebd760498689c63a64473c665c" member_type: VOTER }
I20251210 21:04:21.212543 24144 raft_consensus.cc:385] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:04:21.212576 24144 raft_consensus.cc:740] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9294ebd760498689c63a64473c665c, State: Initialized, Role: FOLLOWER
I20251210 21:04:21.212733 24144 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9294ebd760498689c63a64473c665c" member_type: VOTER }
I20251210 21:04:21.212842 24144 raft_consensus.cc:399] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:04:21.212900 24144 raft_consensus.cc:493] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:04:21.212967 24144 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:04:21.214576 24144 raft_consensus.cc:515] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9294ebd760498689c63a64473c665c" member_type: VOTER }
I20251210 21:04:21.214741 24144 leader_election.cc:304] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef9294ebd760498689c63a64473c665c; no voters:
I20251210 21:04:21.215122 24144 leader_election.cc:290] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:04:21.215265 24147 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:04:21.215777 24147 raft_consensus.cc:697] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 1 LEADER]: Becoming Leader. State: Replica: ef9294ebd760498689c63a64473c665c, State: Running, Role: LEADER
I20251210 21:04:21.215909 24144 sys_catalog.cc:565] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:04:21.216012 24147 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9294ebd760498689c63a64473c665c" member_type: VOTER }
I20251210 21:04:21.216720 24149 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef9294ebd760498689c63a64473c665c. Latest consensus state: current_term: 1 leader_uuid: "ef9294ebd760498689c63a64473c665c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9294ebd760498689c63a64473c665c" member_type: VOTER } }
I20251210 21:04:21.216797 24148 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef9294ebd760498689c63a64473c665c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9294ebd760498689c63a64473c665c" member_type: VOTER } }
I20251210 21:04:21.216940 24149 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:21.217026 24148 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:21.218089 24152 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:04:21.219374 24152 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:04:21.221191 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:04:21.222549 24152 catalog_manager.cc:1357] Generated new cluster ID: ada5f0fd2cde457bb861159017d86a4f
I20251210 21:04:21.222657 24152 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:04:21.231101 24152 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:04:21.231957 24152 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:04:21.238937 24152 catalog_manager.cc:6027] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c: Generated new TSK 0
I20251210 21:04:21.239164 24152 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:04:21.253623 22013 internal_mini_cluster.cc:371] 0 TS(s) registered with all masters after 8.846e-05s
I20251210 21:04:33.336145 22013 master.cc:561] Master@127.21.127.126:40343 shutting down...
I20251210 21:04:33.348753 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:04:33.349021 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:33.349126 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P ef9294ebd760498689c63a64473c665c: stopping tablet replica
I20251210 21:04:33.363150 22013 master.cc:583] Master@127.21.127.126:40343 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTservers (12206 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTablets
I20251210 21:04:33.371007 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:40103
I20251210 21:04:33.371568 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:33.373746 24166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.373963 24167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.374233 24169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:33.376744 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:33.378330 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:33.378399 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:33.378435 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400673378421 us; error 0 us; skew 500 ppm
I20251210 21:04:33.378567 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:33.379805 22013 webserver.cc:492] Webserver started at http://127.21.127.126:38863/ using document root <none> and password file <none>
I20251210 21:04:33.380028 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:33.380090 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:33.380199 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:33.380777 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "fd49ad68db43408ba7e67cf77573b0c8"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.383141 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:04:33.384687 24174 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.385031 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:33.385123 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "fd49ad68db43408ba7e67cf77573b0c8"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.385226 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:33.391760 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:33.392184 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:33.402472 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:40103
I20251210 21:04:33.402490 24225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:40103 every 8 connection(s)
I20251210 21:04:33.403767 24226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:33.407848 24226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8: Bootstrap starting.
I20251210 21:04:33.409153 24226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8: Neither blocks nor log segments found. Creating new log.
I20251210 21:04:33.410630 24226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8: No bootstrap required, opened a new log
I20251210 21:04:33.411234 24226 raft_consensus.cc:359] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd49ad68db43408ba7e67cf77573b0c8" member_type: VOTER }
I20251210 21:04:33.411375 24226 raft_consensus.cc:385] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:04:33.411468 24226 raft_consensus.cc:740] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd49ad68db43408ba7e67cf77573b0c8, State: Initialized, Role: FOLLOWER
I20251210 21:04:33.411700 24226 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd49ad68db43408ba7e67cf77573b0c8" member_type: VOTER }
I20251210 21:04:33.411831 24226 raft_consensus.cc:399] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:04:33.411904 24226 raft_consensus.cc:493] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:04:33.412002 24226 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:04:33.413710 24226 raft_consensus.cc:515] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd49ad68db43408ba7e67cf77573b0c8" member_type: VOTER }
I20251210 21:04:33.413902 24226 leader_election.cc:304] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd49ad68db43408ba7e67cf77573b0c8; no voters:
I20251210 21:04:33.414326 24226 leader_election.cc:290] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:04:33.414439 24229 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:04:33.414927 24229 raft_consensus.cc:697] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 1 LEADER]: Becoming Leader. State: Replica: fd49ad68db43408ba7e67cf77573b0c8, State: Running, Role: LEADER
I20251210 21:04:33.415153 24226 sys_catalog.cc:565] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:04:33.415154 24229 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd49ad68db43408ba7e67cf77573b0c8" member_type: VOTER }
I20251210 21:04:33.416054 24230 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd49ad68db43408ba7e67cf77573b0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd49ad68db43408ba7e67cf77573b0c8" member_type: VOTER } }
I20251210 21:04:33.416131 24231 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd49ad68db43408ba7e67cf77573b0c8. Latest consensus state: current_term: 1 leader_uuid: "fd49ad68db43408ba7e67cf77573b0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd49ad68db43408ba7e67cf77573b0c8" member_type: VOTER } }
I20251210 21:04:33.416281 24230 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:33.416308 24231 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:33.417259 24234 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:04:33.418681 24234 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:04:33.420289 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:04:33.421754 24234 catalog_manager.cc:1357] Generated new cluster ID: 368bb4f92bbe437bb214fb8e6c42a4fe
I20251210 21:04:33.421838 24234 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:04:33.435708 24234 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:04:33.436456 24234 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:04:33.443867 24234 catalog_manager.cc:6027] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8: Generated new TSK 0
I20251210 21:04:33.444080 24234 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:04:33.453049 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:33.455802 24247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.456218 24248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.456487 24250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:33.456823 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:33.457024 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:33.457090 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:33.457144 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400673457125 us; error 0 us; skew 500 ppm
I20251210 21:04:33.457299 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:33.458380 22013 webserver.cc:492] Webserver started at http://127.21.127.65:45211/ using document root <none> and password file <none>
I20251210 21:04:33.458602 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:33.458693 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:33.458832 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:33.459507 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "c4b5b1ab29124a9baf1f06f91e31d771"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.462005 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251210 21:04:33.463567 24255 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.463928 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:04:33.464028 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "c4b5b1ab29124a9baf1f06f91e31d771"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.464145 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:33.480324 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:33.480942 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:33.481720 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:33.482635 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:33.482698 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.482767 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:33.482810 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.495290 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:44777
I20251210 21:04:33.495311 24317 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:44777 every 8 connection(s)
I20251210 21:04:33.496912 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:33.500312 24323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.500411 24322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.500886 24325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:33.502207 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:33.502486 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:33.502588 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:33.502689 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400673502665 us; error 0 us; skew 500 ppm
I20251210 21:04:33.502867 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:33.503980 22013 webserver.cc:492] Webserver started at http://127.21.127.66:37243/ using document root <none> and password file <none>
I20251210 21:04:33.504184 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:33.504254 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:33.504366 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:33.505039 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "c18d64ca840b4f1c9633078a3299f1a2"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.507397 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:04:33.509307 24330 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.509454 24318 heartbeater.cc:344] Connected to a master server at 127.21.127.126:40103
I20251210 21:04:33.509603 24318 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:33.509747 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251210 21:04:33.509851 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "c18d64ca840b4f1c9633078a3299f1a2"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.509959 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:33.510155 24318 heartbeater.cc:507] Master 127.21.127.126:40103 requested a full tablet report, sending...
I20251210 21:04:33.511204 24191 ts_manager.cc:194] Registered new tserver with Master: c4b5b1ab29124a9baf1f06f91e31d771 (127.21.127.65:44777)
I20251210 21:04:33.512916 24191 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45664
I20251210 21:04:33.517727 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:33.518242 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:33.518747 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:33.519615 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:33.519685 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.519778 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:33.519833 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.531509 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:35737
I20251210 21:04:33.531566 24392 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:35737 every 8 connection(s)
I20251210 21:04:33.533128 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:33.536187 24396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.536820 24397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:33.538394 24399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:33.538714 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:33.539019 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:33.539129 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:33.539234 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400673539205 us; error 0 us; skew 500 ppm
I20251210 21:04:33.539405 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:33.540602 24393 heartbeater.cc:344] Connected to a master server at 127.21.127.126:40103
I20251210 21:04:33.540680 22013 webserver.cc:492] Webserver started at http://127.21.127.67:42785/ using document root <none> and password file <none>
I20251210 21:04:33.540747 24393 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:33.540944 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:33.541025 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:33.541165 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:33.541136 24393 heartbeater.cc:507] Master 127.21.127.126:40103 requested a full tablet report, sending...
I20251210 21:04:33.541783 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "bca319c3631f40869326a494223c9949"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.542047 24191 ts_manager.cc:194] Registered new tserver with Master: c18d64ca840b4f1c9633078a3299f1a2 (127.21.127.66:35737)
I20251210 21:04:33.542831 24191 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45668
I20251210 21:04:33.544394 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:04:33.546034 24404 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.546378 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:33.546474 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "bca319c3631f40869326a494223c9949"
format_stamp: "Formatted at 2025-12-10 21:04:33 on dist-test-slave-9pq8"
I20251210 21:04:33.546591 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:33.564994 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:33.565619 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:33.566162 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:33.567075 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:33.567148 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.567241 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:33.567301 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:33.579432 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:44475
I20251210 21:04:33.579480 24466 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:44475 every 8 connection(s)
I20251210 21:04:33.586408 24467 heartbeater.cc:344] Connected to a master server at 127.21.127.126:40103
I20251210 21:04:33.586581 24467 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:33.587002 24467 heartbeater.cc:507] Master 127.21.127.126:40103 requested a full tablet report, sending...
I20251210 21:04:33.587898 24191 ts_manager.cc:194] Registered new tserver with Master: bca319c3631f40869326a494223c9949 (127.21.127.67:44475)
I20251210 21:04:33.588310 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007958827s
I20251210 21:04:33.588752 24191 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45680
I20251210 21:04:34.514313 24318 heartbeater.cc:499] Master 127.21.127.126:40103 was elected leader, sending a full tablet report...
I20251210 21:04:34.544148 24393 heartbeater.cc:499] Master 127.21.127.126:40103 was elected leader, sending a full tablet report...
I20251210 21:04:34.590056 24467 heartbeater.cc:499] Master 127.21.127.126:40103 was elected leader, sending a full tablet report...
I20251210 21:04:35.420145 24244 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:37.420596 24244 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:39.421028 24244 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:41.421468 24244 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:43.421950 24244 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:45.422358 24244 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:45.671229 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:04:45.685523 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:45.699826 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:04:45.702582 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:04:45.710094 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:45.722784 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:04:45.725433 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:04:45.732721 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:45.745484 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:04:45.748088 22013 master.cc:561] Master@127.21.127.126:40103 shutting down...
I20251210 21:04:45.756070 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:04:45.756300 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:45.756390 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P fd49ad68db43408ba7e67cf77573b0c8: stopping tablet replica
I20251210 21:04:45.769676 22013 master.cc:583] Master@127.21.127.126:40103 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTablets (12407 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne
I20251210 21:04:45.778105 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:46491
I20251210 21:04:45.778605 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:45.781111 24471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:45.781153 24472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:45.781153 24474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:45.781577 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:45.782343 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:45.782416 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:45.782470 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400685782452 us; error 0 us; skew 500 ppm
I20251210 21:04:45.782646 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:45.783905 22013 webserver.cc:492] Webserver started at http://127.21.127.126:42933/ using document root <none> and password file <none>
I20251210 21:04:45.784142 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:45.784212 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:45.784353 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:45.785002 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "fc9db58cdf024c1086380d5b67762b58"
format_stamp: "Formatted at 2025-12-10 21:04:45 on dist-test-slave-9pq8"
I20251210 21:04:45.787518 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:04:45.789175 24479 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:45.789505 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:04:45.789625 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "fc9db58cdf024c1086380d5b67762b58"
format_stamp: "Formatted at 2025-12-10 21:04:45 on dist-test-slave-9pq8"
I20251210 21:04:45.789753 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:45.817452 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:45.818089 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:45.830400 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:46491
I20251210 21:04:45.830435 24530 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:46491 every 8 connection(s)
I20251210 21:04:45.831804 24531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:45.835834 24531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58: Bootstrap starting.
I20251210 21:04:45.837145 24531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58: Neither blocks nor log segments found. Creating new log.
I20251210 21:04:45.838758 24531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58: No bootstrap required, opened a new log
I20251210 21:04:45.839367 24531 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9db58cdf024c1086380d5b67762b58" member_type: VOTER }
I20251210 21:04:45.839533 24531 raft_consensus.cc:385] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:04:45.839623 24531 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9db58cdf024c1086380d5b67762b58, State: Initialized, Role: FOLLOWER
I20251210 21:04:45.839839 24531 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9db58cdf024c1086380d5b67762b58" member_type: VOTER }
I20251210 21:04:45.839975 24531 raft_consensus.cc:399] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:04:45.840052 24531 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:04:45.840140 24531 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:04:45.841858 24531 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9db58cdf024c1086380d5b67762b58" member_type: VOTER }
I20251210 21:04:45.842048 24531 leader_election.cc:304] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc9db58cdf024c1086380d5b67762b58; no voters:
I20251210 21:04:45.842430 24531 leader_election.cc:290] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:04:45.842541 24534 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:04:45.843031 24534 raft_consensus.cc:697] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 1 LEADER]: Becoming Leader. State: Replica: fc9db58cdf024c1086380d5b67762b58, State: Running, Role: LEADER
I20251210 21:04:45.843277 24531 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:04:45.843248 24534 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9db58cdf024c1086380d5b67762b58" member_type: VOTER }
I20251210 21:04:45.844143 24535 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc9db58cdf024c1086380d5b67762b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9db58cdf024c1086380d5b67762b58" member_type: VOTER } }
I20251210 21:04:45.844202 24536 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc9db58cdf024c1086380d5b67762b58. Latest consensus state: current_term: 1 leader_uuid: "fc9db58cdf024c1086380d5b67762b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9db58cdf024c1086380d5b67762b58" member_type: VOTER } }
I20251210 21:04:45.844378 24535 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:45.844488 24536 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:45.845389 24539 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:04:45.846848 24539 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:04:45.848304 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:04:45.849970 24539 catalog_manager.cc:1357] Generated new cluster ID: b9d15037ae0c4a428b0af88008d3283e
I20251210 21:04:45.850042 24539 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:04:45.868346 24539 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:04:45.869190 24539 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:04:45.880254 24539 catalog_manager.cc:6027] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58: Generated new TSK 0
I20251210 21:04:45.880486 24539 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:04:45.913420 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:45.916450 24552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:45.916523 24553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:45.916785 24555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:45.917834 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:45.918102 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:45.918166 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:45.918224 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400685918208 us; error 0 us; skew 500 ppm
I20251210 21:04:45.918350 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:45.919591 22013 webserver.cc:492] Webserver started at http://127.21.127.65:34301/ using document root <none> and password file <none>
I20251210 21:04:45.919831 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:45.919901 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:45.920038 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:45.920684 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "6f5f99976fae419fb6d0a07f016781fa"
format_stamp: "Formatted at 2025-12-10 21:04:45 on dist-test-slave-9pq8"
I20251210 21:04:45.923223 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251210 21:04:45.924836 24560 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:45.925208 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:45.925307 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "6f5f99976fae419fb6d0a07f016781fa"
format_stamp: "Formatted at 2025-12-10 21:04:45 on dist-test-slave-9pq8"
I20251210 21:04:45.925416 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:45.937354 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:45.937983 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:45.938498 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:45.939330 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:45.939394 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:45.939497 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:45.939568 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:45.953121 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:35307
I20251210 21:04:45.953154 24622 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:35307 every 8 connection(s)
I20251210 21:04:45.954670 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:45.957825 24627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:45.957857 24628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:45.958433 24630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:45.958623 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:45.958958 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:45.959043 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:45.959090 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400685959070 us; error 0 us; skew 500 ppm
I20251210 21:04:45.959249 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:45.960556 22013 webserver.cc:492] Webserver started at http://127.21.127.66:40459/ using document root <none> and password file <none>
I20251210 21:04:45.960848 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:45.960947 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:45.961102 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:45.961997 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "3285e3ec4c424e76bf3cb104d7fd7574"
format_stamp: "Formatted at 2025-12-10 21:04:45 on dist-test-slave-9pq8"
I20251210 21:04:45.963263 24623 heartbeater.cc:344] Connected to a master server at 127.21.127.126:46491
I20251210 21:04:45.963411 24623 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:45.963961 24623 heartbeater.cc:507] Master 127.21.127.126:46491 requested a full tablet report, sending...
I20251210 21:04:45.965015 24496 ts_manager.cc:194] Registered new tserver with Master: 6f5f99976fae419fb6d0a07f016781fa (127.21.127.65:35307)
I20251210 21:04:45.965233 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20251210 21:04:45.966552 24496 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49830
I20251210 21:04:45.967311 24635 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:45.967682 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:04:45.967792 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "3285e3ec4c424e76bf3cb104d7fd7574"
format_stamp: "Formatted at 2025-12-10 21:04:45 on dist-test-slave-9pq8"
I20251210 21:04:45.967927 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:45.987021 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:45.987668 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:45.988232 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:45.989102 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:45.989241 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:45.989372 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:45.989427 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:46.001235 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:34861
I20251210 21:04:46.001258 24697 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:34861 every 8 connection(s)
I20251210 21:04:46.002779 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:46.005666 24702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:46.005695 24701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:46.007755 24704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:46.008033 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:46.008421 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:46.008498 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:46.008584 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400686008550 us; error 0 us; skew 500 ppm
I20251210 21:04:46.008884 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:46.009671 24698 heartbeater.cc:344] Connected to a master server at 127.21.127.126:46491
I20251210 21:04:46.009814 24698 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:46.010203 24698 heartbeater.cc:507] Master 127.21.127.126:46491 requested a full tablet report, sending...
I20251210 21:04:46.010233 22013 webserver.cc:492] Webserver started at http://127.21.127.67:34373/ using document root <none> and password file <none>
I20251210 21:04:46.010443 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:46.010546 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:46.010701 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:46.011121 24496 ts_manager.cc:194] Registered new tserver with Master: 3285e3ec4c424e76bf3cb104d7fd7574 (127.21.127.66:34861)
I20251210 21:04:46.011406 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "6710f976be1b4cb1b1b7be51fab872e6"
format_stamp: "Formatted at 2025-12-10 21:04:46 on dist-test-slave-9pq8"
I20251210 21:04:46.011976 24496 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49840
I20251210 21:04:46.013899 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:04:46.015399 24709 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:46.015744 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:46.015848 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "6710f976be1b4cb1b1b7be51fab872e6"
format_stamp: "Formatted at 2025-12-10 21:04:46 on dist-test-slave-9pq8"
I20251210 21:04:46.015956 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:46.033742 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:46.034368 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:46.034955 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:46.035847 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:46.035917 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:46.036008 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:46.036065 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:46.048902 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:35515
I20251210 21:04:46.048928 24771 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:35515 every 8 connection(s)
I20251210 21:04:46.050431 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:46.053408 24775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:46.054271 24776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:46.054322 24778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:46.057605 24772 heartbeater.cc:344] Connected to a master server at 127.21.127.126:46491
I20251210 21:04:46.057750 24772 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:46.058194 24772 heartbeater.cc:507] Master 127.21.127.126:46491 requested a full tablet report, sending...
I20251210 21:04:46.059237 24496 ts_manager.cc:194] Registered new tserver with Master: 6710f976be1b4cb1b1b7be51fab872e6 (127.21.127.67:35515)
I20251210 21:04:46.060300 24496 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49856
I20251210 21:04:46.060766 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:46.068753 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:46.068835 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:46.068861 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400686068857 us; error 0 us; skew 500 ppm
I20251210 21:04:46.068984 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:46.070179 22013 webserver.cc:492] Webserver started at http://127.21.127.68:44351/ using document root <none> and password file <none>
I20251210 21:04:46.070370 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:46.070422 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:46.070521 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:46.071074 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "48215a668bf94142859a07a176226595"
format_stamp: "Formatted at 2025-12-10 21:04:46 on dist-test-slave-9pq8"
I20251210 21:04:46.073356 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:04:46.074945 24783 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:46.075286 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:46.075387 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "48215a668bf94142859a07a176226595"
format_stamp: "Formatted at 2025-12-10 21:04:46 on dist-test-slave-9pq8"
I20251210 21:04:46.075527 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:46.104384 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:46.104918 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:46.105460 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:46.106335 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:46.106390 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:46.106457 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:46.106492 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:46.116894 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:37809
I20251210 21:04:46.116951 24845 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:37809 every 8 connection(s)
I20251210 21:04:46.123097 24846 heartbeater.cc:344] Connected to a master server at 127.21.127.126:46491
I20251210 21:04:46.123215 24846 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:46.123543 24846 heartbeater.cc:507] Master 127.21.127.126:46491 requested a full tablet report, sending...
I20251210 21:04:46.124301 24496 ts_manager.cc:194] Registered new tserver with Master: 48215a668bf94142859a07a176226595 (127.21.127.68:37809)
I20251210 21:04:46.124420 22013 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.00669249s
I20251210 21:04:46.125115 24496 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49858
I20251210 21:04:46.968276 24623 heartbeater.cc:499] Master 127.21.127.126:46491 was elected leader, sending a full tablet report...
I20251210 21:04:47.013162 24698 heartbeater.cc:499] Master 127.21.127.126:46491 was elected leader, sending a full tablet report...
I20251210 21:04:47.061524 24772 heartbeater.cc:499] Master 127.21.127.126:46491 was elected leader, sending a full tablet report...
I20251210 21:04:47.126286 24846 heartbeater.cc:499] Master 127.21.127.126:46491 was elected leader, sending a full tablet report...
I20251210 21:04:47.159176 22013 test_util.cc:276] Using random seed: 1624778610
I20251210 21:04:47.170797 24496 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49862:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251210 21:04:47.186896 24588 tablet_service.cc:1505] Processing CreateTablet for tablet 06b8543b57664c54a498fa228ff81d09 (DEFAULT_TABLE table=test-workload [id=699c33c7304f42c49aeeefd1575e6ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:04:47.187302 24588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b8543b57664c54a498fa228ff81d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:47.191418 24663 tablet_service.cc:1505] Processing CreateTablet for tablet 06b8543b57664c54a498fa228ff81d09 (DEFAULT_TABLE table=test-workload [id=699c33c7304f42c49aeeefd1575e6ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:04:47.191947 24663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b8543b57664c54a498fa228ff81d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:47.192086 24737 tablet_service.cc:1505] Processing CreateTablet for tablet 06b8543b57664c54a498fa228ff81d09 (DEFAULT_TABLE table=test-workload [id=699c33c7304f42c49aeeefd1575e6ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:04:47.192394 24737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b8543b57664c54a498fa228ff81d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:47.196055 24867 tablet_bootstrap.cc:492] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574: Bootstrap starting.
I20251210 21:04:47.197703 24869 tablet_bootstrap.cc:492] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa: Bootstrap starting.
I20251210 21:04:47.198009 24868 tablet_bootstrap.cc:492] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6: Bootstrap starting.
I20251210 21:04:47.198300 24867 tablet_bootstrap.cc:654] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574: Neither blocks nor log segments found. Creating new log.
I20251210 21:04:47.199417 24869 tablet_bootstrap.cc:654] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa: Neither blocks nor log segments found. Creating new log.
I20251210 21:04:47.199417 24868 tablet_bootstrap.cc:654] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6: Neither blocks nor log segments found. Creating new log.
I20251210 21:04:47.200824 24867 tablet_bootstrap.cc:492] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574: No bootstrap required, opened a new log
I20251210 21:04:47.200937 24867 ts_tablet_manager.cc:1403] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20251210 21:04:47.201059 24869 tablet_bootstrap.cc:492] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa: No bootstrap required, opened a new log
I20251210 21:04:47.201170 24869 ts_tablet_manager.cc:1403] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251210 21:04:47.201543 24867 raft_consensus.cc:359] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.201684 24867 raft_consensus.cc:385] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:04:47.201719 24869 raft_consensus.cc:359] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.201736 24867 raft_consensus.cc:740] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3285e3ec4c424e76bf3cb104d7fd7574, State: Initialized, Role: FOLLOWER
I20251210 21:04:47.201869 24869 raft_consensus.cc:385] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:04:47.201942 24869 raft_consensus.cc:740] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f5f99976fae419fb6d0a07f016781fa, State: Initialized, Role: FOLLOWER
I20251210 21:04:47.202111 24869 consensus_queue.cc:260] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.202405 24868 tablet_bootstrap.cc:492] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6: No bootstrap required, opened a new log
I20251210 21:04:47.202529 24868 ts_tablet_manager.cc:1403] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251210 21:04:47.202711 24869 ts_tablet_manager.cc:1434] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251210 21:04:47.203204 24868 raft_consensus.cc:359] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.203318 24868 raft_consensus.cc:385] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:04:47.203359 24868 raft_consensus.cc:740] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6710f976be1b4cb1b1b7be51fab872e6, State: Initialized, Role: FOLLOWER
I20251210 21:04:47.203554 24867 consensus_queue.cc:260] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.203554 24868 consensus_queue.cc:260] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.204627 24867 ts_tablet_manager.cc:1434] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251210 21:04:47.204931 24868 ts_tablet_manager.cc:1434] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20251210 21:04:47.205072 24624 tablet.cc:2378] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251210 21:04:47.484865 24873 raft_consensus.cc:493] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:04:47.485041 24873 raft_consensus.cc:515] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.485884 24873 leader_election.cc:290] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3285e3ec4c424e76bf3cb104d7fd7574 (127.21.127.66:34861), 6710f976be1b4cb1b1b7be51fab872e6 (127.21.127.67:35515)
I20251210 21:04:47.491675 24747 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8543b57664c54a498fa228ff81d09" candidate_uuid: "6f5f99976fae419fb6d0a07f016781fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710f976be1b4cb1b1b7be51fab872e6" is_pre_election: true
I20251210 21:04:47.491678 24673 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8543b57664c54a498fa228ff81d09" candidate_uuid: "6f5f99976fae419fb6d0a07f016781fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" is_pre_election: true
I20251210 21:04:47.491945 24747 raft_consensus.cc:2468] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f5f99976fae419fb6d0a07f016781fa in term 0.
I20251210 21:04:47.491956 24673 raft_consensus.cc:2468] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f5f99976fae419fb6d0a07f016781fa in term 0.
I20251210 21:04:47.492360 24561 leader_election.cc:304] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6710f976be1b4cb1b1b7be51fab872e6, 6f5f99976fae419fb6d0a07f016781fa; no voters:
I20251210 21:04:47.492614 24873 raft_consensus.cc:2804] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:04:47.492681 24873 raft_consensus.cc:493] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:04:47.492849 24873 raft_consensus.cc:3060] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:04:47.494622 24873 raft_consensus.cc:515] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.495136 24873 leader_election.cc:290] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [CANDIDATE]: Term 1 election: Requested vote from peers 3285e3ec4c424e76bf3cb104d7fd7574 (127.21.127.66:34861), 6710f976be1b4cb1b1b7be51fab872e6 (127.21.127.67:35515)
I20251210 21:04:47.495544 24747 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8543b57664c54a498fa228ff81d09" candidate_uuid: "6f5f99976fae419fb6d0a07f016781fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710f976be1b4cb1b1b7be51fab872e6"
I20251210 21:04:47.495599 24673 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8543b57664c54a498fa228ff81d09" candidate_uuid: "6f5f99976fae419fb6d0a07f016781fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3285e3ec4c424e76bf3cb104d7fd7574"
I20251210 21:04:47.495673 24747 raft_consensus.cc:3060] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:04:47.495743 24673 raft_consensus.cc:3060] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:04:47.497433 24673 raft_consensus.cc:2468] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f5f99976fae419fb6d0a07f016781fa in term 1.
I20251210 21:04:47.497452 24747 raft_consensus.cc:2468] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f5f99976fae419fb6d0a07f016781fa in term 1.
I20251210 21:04:47.497795 24564 leader_election.cc:304] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3285e3ec4c424e76bf3cb104d7fd7574, 6710f976be1b4cb1b1b7be51fab872e6, 6f5f99976fae419fb6d0a07f016781fa; no voters:
I20251210 21:04:47.498018 24873 raft_consensus.cc:2804] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:04:47.498337 24873 raft_consensus.cc:697] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f5f99976fae419fb6d0a07f016781fa, State: Running, Role: LEADER
I20251210 21:04:47.498554 24873 consensus_queue.cc:237] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } }
I20251210 21:04:47.500865 24496 catalog_manager.cc:5654] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f5f99976fae419fb6d0a07f016781fa (127.21.127.65). New cstate: current_term: 1 leader_uuid: "6f5f99976fae419fb6d0a07f016781fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f5f99976fae419fb6d0a07f016781fa" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 35307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 } health_report { overall_health: UNKNOWN } } }
I20251210 21:04:47.848387 24549 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:47.848934 24549 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:47.849056 24549 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:48.015275 24880 consensus_queue.cc:1048] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6710f976be1b4cb1b1b7be51fab872e6" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:04:48.020699 24887 consensus_queue.cc:1048] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3285e3ec4c424e76bf3cb104d7fd7574" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 34861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:04:49.849455 24549 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:49.849938 24549 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:49.850047 24549 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:51.850456 24549 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:51.850979 24549 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:51.851090 24549 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:53.851527 24549 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:53.852016 24549 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:53.852133 24549 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:55.852543 24549 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:55.853060 24549 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:55.853168 24549 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:57.853622 24549 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:04:57.854157 24549 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:04:57.854295 24549 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:04:58.604211 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:04:58.618345 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:58.618695 22013 tablet_replica.cc:333] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa: stopping tablet replica
I20251210 21:04:58.618880 22013 raft_consensus.cc:2243] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:04:58.619226 22013 raft_consensus.cc:2272] T 06b8543b57664c54a498fa228ff81d09 P 6f5f99976fae419fb6d0a07f016781fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:58.633380 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:04:58.636569 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:04:58.644268 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:58.644523 22013 tablet_replica.cc:333] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574: stopping tablet replica
I20251210 21:04:58.644783 22013 raft_consensus.cc:2243] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:04:58.644991 22013 raft_consensus.cc:2272] T 06b8543b57664c54a498fa228ff81d09 P 3285e3ec4c424e76bf3cb104d7fd7574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:58.658425 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:04:58.661538 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:04:58.668923 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:58.669274 22013 tablet_replica.cc:333] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6: stopping tablet replica
I20251210 21:04:58.669458 22013 raft_consensus.cc:2243] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:04:58.669682 22013 raft_consensus.cc:2272] T 06b8543b57664c54a498fa228ff81d09 P 6710f976be1b4cb1b1b7be51fab872e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:58.683110 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:04:58.686422 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:04:58.693836 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:04:58.706658 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:04:58.720911 22013 master.cc:561] Master@127.21.127.126:46491 shutting down...
I20251210 21:04:58.728545 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:04:58.728804 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:04:58.728904 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P fc9db58cdf024c1086380d5b67762b58: stopping tablet replica
I20251210 21:04:58.742301 22013 master.cc:583] Master@127.21.127.126:46491 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne (12975 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy
I20251210 21:04:58.753355 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:34055
I20251210 21:04:58.753872 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:58.756275 24891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.756354 24892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.756932 24894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:58.757231 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:58.757448 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:58.757511 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:58.757545 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400698757542 us; error 0 us; skew 500 ppm
I20251210 21:04:58.757686 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:58.758957 22013 webserver.cc:492] Webserver started at http://127.21.127.126:42787/ using document root <none> and password file <none>
I20251210 21:04:58.759169 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:58.759225 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:58.759346 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:58.760037 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "3ff87f3f9833445cad2fe535ff4d46ab"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.762461 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:04:58.764073 24899 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.764400 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:04:58.764497 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "3ff87f3f9833445cad2fe535ff4d46ab"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.764598 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:58.790318 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:58.790882 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:58.803007 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:34055
I20251210 21:04:58.803046 24950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:34055 every 8 connection(s)
I20251210 21:04:58.804335 24951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:04:58.808471 24951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab: Bootstrap starting.
I20251210 21:04:58.809741 24951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab: Neither blocks nor log segments found. Creating new log.
I20251210 21:04:58.811389 24951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab: No bootstrap required, opened a new log
I20251210 21:04:58.812016 24951 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" member_type: VOTER }
I20251210 21:04:58.812148 24951 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:04:58.812189 24951 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff87f3f9833445cad2fe535ff4d46ab, State: Initialized, Role: FOLLOWER
I20251210 21:04:58.812381 24951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" member_type: VOTER }
I20251210 21:04:58.812489 24951 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:04:58.812531 24951 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:04:58.812583 24951 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:04:58.814340 24951 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" member_type: VOTER }
I20251210 21:04:58.814518 24951 leader_election.cc:304] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ff87f3f9833445cad2fe535ff4d46ab; no voters:
I20251210 21:04:58.814880 24951 leader_election.cc:290] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:04:58.815032 24954 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:04:58.815500 24954 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 1 LEADER]: Becoming Leader. State: Replica: 3ff87f3f9833445cad2fe535ff4d46ab, State: Running, Role: LEADER
I20251210 21:04:58.815728 24954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" member_type: VOTER }
I20251210 21:04:58.815765 24951 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:04:58.816491 24956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ff87f3f9833445cad2fe535ff4d46ab. Latest consensus state: current_term: 1 leader_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" member_type: VOTER } }
I20251210 21:04:58.816541 24955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff87f3f9833445cad2fe535ff4d46ab" member_type: VOTER } }
I20251210 21:04:58.816664 24956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:58.816701 24955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [sys.catalog]: This master's current role is: LEADER
I20251210 21:04:58.817972 24959 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:04:58.819972 24959 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:04:58.820762 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:04:58.823232 24959 catalog_manager.cc:1357] Generated new cluster ID: cec5da9c6c8e44ce8743771ffde3a8ef
I20251210 21:04:58.823313 24959 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:04:58.845368 24959 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:04:58.846201 24959 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:04:58.853542 24959 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab: Generated new TSK 0
I20251210 21:04:58.853801 24959 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:04:58.885814 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:58.888708 24972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.888757 24973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.889187 24975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:58.889370 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:58.889576 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:58.889631 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:58.889652 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400698889647 us; error 0 us; skew 500 ppm
I20251210 21:04:58.889787 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:58.890861 22013 webserver.cc:492] Webserver started at http://127.21.127.65:45859/ using document root <none> and password file <none>
I20251210 21:04:58.891088 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:58.891155 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:58.891271 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:58.891969 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "ef2e9185f01a432482bfb9c4cca5dab7"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.894354 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:04:58.895980 24980 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.896353 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:58.896435 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "ef2e9185f01a432482bfb9c4cca5dab7"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.896528 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:58.912626 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:58.913154 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:58.913621 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:58.914490 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:58.914541 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.914616 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:58.914654 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.925668 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:45079
I20251210 21:04:58.925680 25042 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:45079 every 8 connection(s)
I20251210 21:04:58.927126 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:58.929972 25048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.929991 25047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.930313 25050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:58.930804 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:58.931059 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:58.931110 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:58.931133 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400698931128 us; error 0 us; skew 500 ppm
I20251210 21:04:58.931254 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:58.932775 22013 webserver.cc:492] Webserver started at http://127.21.127.66:46077/ using document root <none> and password file <none>
I20251210 21:04:58.933044 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:58.933121 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:58.933235 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:58.933550 25043 heartbeater.cc:344] Connected to a master server at 127.21.127.126:34055
I20251210 21:04:58.933663 25043 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:58.933969 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "14d9a9d195b1433996ece94d2e6cacca"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.934032 25043 heartbeater.cc:507] Master 127.21.127.126:34055 requested a full tablet report, sending...
I20251210 21:04:58.934897 24916 ts_manager.cc:194] Registered new tserver with Master: ef2e9185f01a432482bfb9c4cca5dab7 (127.21.127.65:45079)
I20251210 21:04:58.935961 24916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48376
I20251210 21:04:58.936622 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:04:58.938271 25055 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.938572 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:58.938656 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "14d9a9d195b1433996ece94d2e6cacca"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.938747 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:58.947362 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:58.947850 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:58.948334 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:58.949164 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:58.949216 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.949266 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:58.949306 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.959769 25117 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:37735 every 8 connection(s)
I20251210 21:04:58.959764 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:37735
I20251210 21:04:58.961180 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:04:58.963868 25122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.964419 25121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:04:58.964496 25124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:04:58.965163 22013 server_base.cc:1047] running on GCE node
I20251210 21:04:58.965528 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:04:58.965600 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:04:58.965642 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400698965638 us; error 0 us; skew 500 ppm
I20251210 21:04:58.965742 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:04:58.967054 22013 webserver.cc:492] Webserver started at http://127.21.127.67:35857/ using document root <none> and password file <none>
I20251210 21:04:58.967272 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:04:58.967362 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:04:58.967523 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:04:58.967617 25118 heartbeater.cc:344] Connected to a master server at 127.21.127.126:34055
I20251210 21:04:58.967751 25118 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:58.968086 25118 heartbeater.cc:507] Master 127.21.127.126:34055 requested a full tablet report, sending...
I20251210 21:04:58.968257 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "5ff9f11b573343f59d333e2e37227d30"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.968951 24916 ts_manager.cc:194] Registered new tserver with Master: 14d9a9d195b1433996ece94d2e6cacca (127.21.127.66:37735)
I20251210 21:04:58.969960 24916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48384
I20251210 21:04:58.971148 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20251210 21:04:58.972770 25129 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.973088 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:04:58.973186 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "5ff9f11b573343f59d333e2e37227d30"
format_stamp: "Formatted at 2025-12-10 21:04:58 on dist-test-slave-9pq8"
I20251210 21:04:58.973299 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:04:58.978559 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:04:58.978986 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:04:58.979470 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:04:58.980250 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:04:58.980316 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.980401 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:04:58.980456 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:04:58.990751 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:38985
I20251210 21:04:58.990782 25191 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:38985 every 8 connection(s)
I20251210 21:04:58.997084 25192 heartbeater.cc:344] Connected to a master server at 127.21.127.126:34055
I20251210 21:04:58.997222 25192 heartbeater.cc:461] Registering TS with master...
I20251210 21:04:58.997543 25192 heartbeater.cc:507] Master 127.21.127.126:34055 requested a full tablet report, sending...
I20251210 21:04:58.998436 24916 ts_manager.cc:194] Registered new tserver with Master: 5ff9f11b573343f59d333e2e37227d30 (127.21.127.67:38985)
I20251210 21:04:58.998502 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00692221s
I20251210 21:04:58.999475 24916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48388
I20251210 21:04:59.937348 25043 heartbeater.cc:499] Master 127.21.127.126:34055 was elected leader, sending a full tablet report...
I20251210 21:04:59.971252 25118 heartbeater.cc:499] Master 127.21.127.126:34055 was elected leader, sending a full tablet report...
I20251210 21:05:00.000644 25192 heartbeater.cc:499] Master 127.21.127.126:34055 was elected leader, sending a full tablet report...
I20251210 21:05:00.034439 22013 test_util.cc:276] Using random seed: 1637653888
I20251210 21:05:00.045176 24916 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48398:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:05:00.045787 24916 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:05:00.063777 25008 tablet_service.cc:1505] Processing CreateTablet for tablet 51318c642b6f49beb89afbb67d9490f8 (DEFAULT_TABLE table=test-workload [id=b0d78c310d95454bacdf6472c2775bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:05:00.064375 25008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51318c642b6f49beb89afbb67d9490f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:00.070191 25083 tablet_service.cc:1505] Processing CreateTablet for tablet 51318c642b6f49beb89afbb67d9490f8 (DEFAULT_TABLE table=test-workload [id=b0d78c310d95454bacdf6472c2775bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:05:00.070392 25157 tablet_service.cc:1505] Processing CreateTablet for tablet 51318c642b6f49beb89afbb67d9490f8 (DEFAULT_TABLE table=test-workload [id=b0d78c310d95454bacdf6472c2775bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:05:00.070617 25083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51318c642b6f49beb89afbb67d9490f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:00.070755 25157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51318c642b6f49beb89afbb67d9490f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:00.075399 25214 tablet_bootstrap.cc:492] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30: Bootstrap starting.
I20251210 21:05:00.075505 25213 tablet_bootstrap.cc:492] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca: Bootstrap starting.
I20251210 21:05:00.076946 25214 tablet_bootstrap.cc:654] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:00.076952 25215 tablet_bootstrap.cc:492] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7: Bootstrap starting.
I20251210 21:05:00.077807 25213 tablet_bootstrap.cc:654] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:00.078603 25214 tablet_bootstrap.cc:492] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30: No bootstrap required, opened a new log
I20251210 21:05:00.078828 25215 tablet_bootstrap.cc:654] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:00.078832 25214 ts_tablet_manager.cc:1403] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:05:00.079526 25213 tablet_bootstrap.cc:492] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca: No bootstrap required, opened a new log
I20251210 21:05:00.079571 25214 raft_consensus.cc:359] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.079743 25213 ts_tablet_manager.cc:1403] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:05:00.079795 25214 raft_consensus.cc:385] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:00.079924 25214 raft_consensus.cc:740] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff9f11b573343f59d333e2e37227d30, State: Initialized, Role: FOLLOWER
I20251210 21:05:00.080222 25214 consensus_queue.cc:260] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.080442 25215 tablet_bootstrap.cc:492] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7: No bootstrap required, opened a new log
I20251210 21:05:00.080619 25215 ts_tablet_manager.cc:1403] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20251210 21:05:00.081187 25213 raft_consensus.cc:359] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.081401 25213 raft_consensus.cc:385] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:00.081382 25215 raft_consensus.cc:359] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.081549 25213 raft_consensus.cc:740] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d9a9d195b1433996ece94d2e6cacca, State: Initialized, Role: FOLLOWER
I20251210 21:05:00.081593 25214 ts_tablet_manager.cc:1434] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251210 21:05:00.081610 25215 raft_consensus.cc:385] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:00.081799 25213 consensus_queue.cc:260] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.081923 25215 raft_consensus.cc:740] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2e9185f01a432482bfb9c4cca5dab7, State: Initialized, Role: FOLLOWER
I20251210 21:05:00.082221 25215 consensus_queue.cc:260] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.082513 25213 ts_tablet_manager.cc:1434] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251210 21:05:00.087823 25215 ts_tablet_manager.cc:1434] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20251210 21:05:00.386513 25220 raft_consensus.cc:493] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:00.386721 25220 raft_consensus.cc:515] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.387569 25220 leader_election.cc:290] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2e9185f01a432482bfb9c4cca5dab7 (127.21.127.65:45079), 5ff9f11b573343f59d333e2e37227d30 (127.21.127.67:38985)
I20251210 21:05:00.393497 25018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51318c642b6f49beb89afbb67d9490f8" candidate_uuid: "14d9a9d195b1433996ece94d2e6cacca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" is_pre_election: true
I20251210 21:05:00.393498 25167 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51318c642b6f49beb89afbb67d9490f8" candidate_uuid: "14d9a9d195b1433996ece94d2e6cacca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff9f11b573343f59d333e2e37227d30" is_pre_election: true
I20251210 21:05:00.393724 25018 raft_consensus.cc:2468] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d9a9d195b1433996ece94d2e6cacca in term 0.
I20251210 21:05:00.393778 25167 raft_consensus.cc:2468] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d9a9d195b1433996ece94d2e6cacca in term 0.
I20251210 21:05:00.394181 25058 leader_election.cc:304] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14d9a9d195b1433996ece94d2e6cacca, 5ff9f11b573343f59d333e2e37227d30, ef2e9185f01a432482bfb9c4cca5dab7; no voters:
I20251210 21:05:00.394431 25220 raft_consensus.cc:2804] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:00.394515 25220 raft_consensus.cc:493] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:00.394603 25220 raft_consensus.cc:3060] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:00.396392 25220 raft_consensus.cc:515] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.396942 25220 leader_election.cc:290] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [CANDIDATE]: Term 1 election: Requested vote from peers ef2e9185f01a432482bfb9c4cca5dab7 (127.21.127.65:45079), 5ff9f11b573343f59d333e2e37227d30 (127.21.127.67:38985)
I20251210 21:05:00.397306 25018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51318c642b6f49beb89afbb67d9490f8" candidate_uuid: "14d9a9d195b1433996ece94d2e6cacca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2e9185f01a432482bfb9c4cca5dab7"
I20251210 21:05:00.397367 25167 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51318c642b6f49beb89afbb67d9490f8" candidate_uuid: "14d9a9d195b1433996ece94d2e6cacca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff9f11b573343f59d333e2e37227d30"
I20251210 21:05:00.397475 25018 raft_consensus.cc:3060] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:00.397526 25167 raft_consensus.cc:3060] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:00.399291 25167 raft_consensus.cc:2468] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d9a9d195b1433996ece94d2e6cacca in term 1.
I20251210 21:05:00.399379 25018 raft_consensus.cc:2468] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d9a9d195b1433996ece94d2e6cacca in term 1.
I20251210 21:05:00.399638 25056 leader_election.cc:304] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d9a9d195b1433996ece94d2e6cacca, 5ff9f11b573343f59d333e2e37227d30; no voters:
I20251210 21:05:00.399891 25220 raft_consensus.cc:2804] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:00.400276 25220 raft_consensus.cc:697] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 1 LEADER]: Becoming Leader. State: Replica: 14d9a9d195b1433996ece94d2e6cacca, State: Running, Role: LEADER
I20251210 21:05:00.400504 25220 consensus_queue.cc:237] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } }
I20251210 21:05:00.402628 24915 catalog_manager.cc:5654] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca reported cstate change: term changed from 0 to 1, leader changed from <none> to 14d9a9d195b1433996ece94d2e6cacca (127.21.127.66). New cstate: current_term: 1 leader_uuid: "14d9a9d195b1433996ece94d2e6cacca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14d9a9d195b1433996ece94d2e6cacca" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 37735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:00.820290 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:00.820708 24969 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:00.821101 24969 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:00.821202 24969 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:00.856554 25220 consensus_queue.cc:1048] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2e9185f01a432482bfb9c4cca5dab7" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:00.862948 25220 consensus_queue.cc:1048] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff9f11b573343f59d333e2e37227d30" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 38985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:01.823562 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:02.821604 24969 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:02.822130 24969 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:02.822252 24969 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:02.825902 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:03.828311 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:04.822633 24969 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:04.823175 24969 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:04.823283 24969 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:04.830683 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:05.833292 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:06.823750 24969 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:06.824208 24969 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:06.824323 24969 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:06.835857 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:07.838327 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:08.824811 24969 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:08.825275 24969 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:08.825371 24969 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:08.840842 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:09.843329 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:10.825788 24969 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:10.826272 24969 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:10.826370 24969 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:10.845839 24964 placement_policy_util.cc:407] tablet 51318c642b6f49beb89afbb67d9490f8: detected majority of replicas (2 of 3) at location L0
I20251210 21:05:11.506289 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:05:11.522359 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:11.522751 22013 tablet_replica.cc:333] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7: stopping tablet replica
I20251210 21:05:11.522933 22013 raft_consensus.cc:2243] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:11.523135 22013 raft_consensus.cc:2272] T 51318c642b6f49beb89afbb67d9490f8 P ef2e9185f01a432482bfb9c4cca5dab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:11.538724 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:05:11.542074 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:05:11.551215 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:11.551512 22013 tablet_replica.cc:333] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca: stopping tablet replica
I20251210 21:05:11.551693 22013 raft_consensus.cc:2243] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:05:11.552049 22013 raft_consensus.cc:2272] T 51318c642b6f49beb89afbb67d9490f8 P 14d9a9d195b1433996ece94d2e6cacca [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:11.565526 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:05:11.568869 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:05:11.576637 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:11.576918 22013 tablet_replica.cc:333] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30: stopping tablet replica
I20251210 21:05:11.577093 22013 raft_consensus.cc:2243] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:11.577272 22013 raft_consensus.cc:2272] T 51318c642b6f49beb89afbb67d9490f8 P 5ff9f11b573343f59d333e2e37227d30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:11.590566 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:05:11.593559 22013 master.cc:561] Master@127.21.127.126:34055 shutting down...
I20251210 21:05:11.601063 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:05:11.601285 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:11.601397 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3ff87f3f9833445cad2fe535ff4d46ab: stopping tablet replica
I20251210 21:05:11.614699 22013 master.cc:583] Master@127.21.127.126:34055 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy (12871 ms)
[ RUN ] AutoRebalancerTest.TestMaxMovesPerServer
I20251210 21:05:11.624651 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:44841
I20251210 21:05:11.625172 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:11.627434 25238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:11.627473 25239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:11.628302 25241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:11.628410 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:11.628719 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:11.628801 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:11.628857 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400711628839 us; error 0 us; skew 500 ppm
I20251210 21:05:11.629040 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:11.630250 22013 webserver.cc:492] Webserver started at http://127.21.127.126:37167/ using document root <none> and password file <none>
I20251210 21:05:11.630481 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:11.630565 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:11.630697 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:11.631373 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "afb8033a8b314114b91c396d6b719bca"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.633764 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:11.635262 25246 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.635645 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:11.635742 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "afb8033a8b314114b91c396d6b719bca"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.635864 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:11.652546 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:11.653046 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:11.664547 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:44841
I20251210 21:05:11.664575 25297 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:44841 every 8 connection(s)
I20251210 21:05:11.665855 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:11.669843 25298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca: Bootstrap starting.
I20251210 21:05:11.670996 25298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:11.672448 25298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca: No bootstrap required, opened a new log
I20251210 21:05:11.673053 25298 raft_consensus.cc:359] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb8033a8b314114b91c396d6b719bca" member_type: VOTER }
I20251210 21:05:11.673183 25298 raft_consensus.cc:385] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:11.673254 25298 raft_consensus.cc:740] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb8033a8b314114b91c396d6b719bca, State: Initialized, Role: FOLLOWER
I20251210 21:05:11.673446 25298 consensus_queue.cc:260] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb8033a8b314114b91c396d6b719bca" member_type: VOTER }
I20251210 21:05:11.673565 25298 raft_consensus.cc:399] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:05:11.673642 25298 raft_consensus.cc:493] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:05:11.673719 25298 raft_consensus.cc:3060] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:11.675362 25298 raft_consensus.cc:515] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb8033a8b314114b91c396d6b719bca" member_type: VOTER }
I20251210 21:05:11.675577 25298 leader_election.cc:304] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afb8033a8b314114b91c396d6b719bca; no voters:
I20251210 21:05:11.675943 25298 leader_election.cc:290] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:05:11.676054 25301 raft_consensus.cc:2804] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:11.676448 25301 raft_consensus.cc:697] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 1 LEADER]: Becoming Leader. State: Replica: afb8033a8b314114b91c396d6b719bca, State: Running, Role: LEADER
I20251210 21:05:11.676704 25298 sys_catalog.cc:565] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:05:11.676663 25301 consensus_queue.cc:237] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb8033a8b314114b91c396d6b719bca" member_type: VOTER }
I20251210 21:05:11.677762 25303 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader afb8033a8b314114b91c396d6b719bca. Latest consensus state: current_term: 1 leader_uuid: "afb8033a8b314114b91c396d6b719bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb8033a8b314114b91c396d6b719bca" member_type: VOTER } }
I20251210 21:05:11.677934 25303 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [sys.catalog]: This master's current role is: LEADER
I20251210 21:05:11.677728 25302 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afb8033a8b314114b91c396d6b719bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb8033a8b314114b91c396d6b719bca" member_type: VOTER } }
I20251210 21:05:11.678202 25302 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [sys.catalog]: This master's current role is: LEADER
I20251210 21:05:11.679047 25307 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:05:11.680428 25307 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:05:11.681795 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:05:11.683182 25307 catalog_manager.cc:1357] Generated new cluster ID: 18a2477870254eeaa18e477f38d1dab3
I20251210 21:05:11.683274 25307 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:05:11.706244 25307 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:05:11.707134 25307 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:05:11.721853 25307 catalog_manager.cc:6027] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca: Generated new TSK 0
I20251210 21:05:11.722118 25307 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:05:11.746914 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:11.749958 25320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:11.749959 25319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:11.750442 25322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:11.750813 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:11.751040 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:11.751102 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:11.751156 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400711751138 us; error 0 us; skew 500 ppm
I20251210 21:05:11.751478 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:11.752686 22013 webserver.cc:492] Webserver started at http://127.21.127.65:41529/ using document root <none> and password file <none>
I20251210 21:05:11.752926 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:11.753000 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:11.753129 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:11.753801 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.756560 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251210 21:05:11.758179 25327 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.758507 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:11.758631 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.758751 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:11.775159 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:11.775794 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:11.776371 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:11.777294 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:11.777364 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.777484 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:11.777537 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.789903 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:41411
I20251210 21:05:11.789930 25389 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:41411 every 8 connection(s)
I20251210 21:05:11.791419 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:11.794296 25395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:11.794309 25394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:11.795729 22013 server_base.cc:1047] running on GCE node
W20251210 21:05:11.795743 25397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:11.796123 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:11.796196 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:11.796257 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400711796235 us; error 0 us; skew 500 ppm
I20251210 21:05:11.796407 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:11.798005 22013 webserver.cc:492] Webserver started at http://127.21.127.66:43329/ using document root <none> and password file <none>
I20251210 21:05:11.798275 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:11.798380 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:11.798527 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:11.798602 25390 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44841
I20251210 21:05:11.798728 25390 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:11.799100 25390 heartbeater.cc:507] Master 127.21.127.126:44841 requested a full tablet report, sending...
I20251210 21:05:11.799273 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "4af27de48bec4590bfcd8d71056814eb"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.800041 25263 ts_manager.cc:194] Registered new tserver with Master: 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:11.801138 25263 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48772
I20251210 21:05:11.801888 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251210 21:05:11.803408 25402 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.803781 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:05:11.803885 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "4af27de48bec4590bfcd8d71056814eb"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.803992 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:11.842000 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:11.842507 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:11.843052 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:11.843866 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:11.843922 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.843991 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:11.844025 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.855180 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:39981
I20251210 21:05:11.855208 25464 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:39981 every 8 connection(s)
I20251210 21:05:11.856596 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:11.859416 25469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:11.859628 25471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:11.859643 25468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:11.859964 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:11.860446 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:11.860519 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:11.860556 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400711860553 us; error 0 us; skew 500 ppm
I20251210 21:05:11.860685 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:11.861943 22013 webserver.cc:492] Webserver started at http://127.21.127.67:36947/ using document root <none> and password file <none>
I20251210 21:05:11.862141 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:11.862201 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:11.862311 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:11.863046 25465 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44841
I20251210 21:05:11.863008 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.863165 25465 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:11.863543 25465 heartbeater.cc:507] Master 127.21.127.126:44841 requested a full tablet report, sending...
I20251210 21:05:11.864454 25263 ts_manager.cc:194] Registered new tserver with Master: 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:11.865221 25263 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48786
I20251210 21:05:11.865643 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:11.867205 25476 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.867588 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:05:11.867684 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
format_stamp: "Formatted at 2025-12-10 21:05:11 on dist-test-slave-9pq8"
I20251210 21:05:11.867794 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:11.885759 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:11.886296 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:11.886831 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:11.887652 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:11.887719 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.887795 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:11.887835 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:11.897738 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:36213
I20251210 21:05:11.897758 25538 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:36213 every 8 connection(s)
I20251210 21:05:11.904058 25539 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44841
I20251210 21:05:11.904209 25539 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:11.904564 25539 heartbeater.cc:507] Master 127.21.127.126:44841 requested a full tablet report, sending...
I20251210 21:05:11.905690 25263 ts_manager.cc:194] Registered new tserver with Master: 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:11.906473 25263 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48800
I20251210 21:05:11.906589 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007984864s
I20251210 21:05:12.802628 25390 heartbeater.cc:499] Master 127.21.127.126:44841 was elected leader, sending a full tablet report...
I20251210 21:05:12.866515 25465 heartbeater.cc:499] Master 127.21.127.126:44841 was elected leader, sending a full tablet report...
I20251210 21:05:12.907788 25539 heartbeater.cc:499] Master 127.21.127.126:44841 was elected leader, sending a full tablet report...
I20251210 21:05:12.934540 22013 test_util.cc:276] Using random seed: 1650553971
I20251210 21:05:12.950299 25263 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48804:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu""\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:05:12.951226 25263 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:05:12.987383 25430 tablet_service.cc:1505] Processing CreateTablet for tablet b13da827577e49569e304f0cbfb5e9df (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251210 21:05:12.987769 25425 tablet_service.cc:1505] Processing CreateTablet for tablet 4f1698d0d3bd4765bc0897771cb0ceee (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251210 21:05:12.987972 25430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b13da827577e49569e304f0cbfb5e9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.988169 25425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f1698d0d3bd4765bc0897771cb0ceee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.988088 25429 tablet_service.cc:1505] Processing CreateTablet for tablet 544270c4ded24ee2ae32f6aa4ec505c2 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251210 21:05:12.988593 25429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544270c4ded24ee2ae32f6aa4ec505c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.991330 25424 tablet_service.cc:1505] Processing CreateTablet for tablet b65abf7cc0484fcda919a741e23928a7 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251210 21:05:12.991722 25424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65abf7cc0484fcda919a741e23928a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.991794 25428 tablet_service.cc:1505] Processing CreateTablet for tablet e7e5750d29e94bad943f054f32b40746 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251210 21:05:12.992112 25428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e5750d29e94bad943f054f32b40746. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.995087 25427 tablet_service.cc:1505] Processing CreateTablet for tablet 4810894f161642aba143677e92893e72 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251210 21:05:12.995515 25427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4810894f161642aba143677e92893e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.995769 25426 tablet_service.cc:1505] Processing CreateTablet for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251210 21:05:12.996158 25426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.996095 25422 tablet_service.cc:1505] Processing CreateTablet for tablet 681d997613474450ba6b9ee3de0f3f24 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251210 21:05:12.996459 25422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681d997613474450ba6b9ee3de0f3f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.997898 25421 tablet_service.cc:1505] Processing CreateTablet for tablet 05b90891884a4db3bc8723b604cec5ea (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251210 21:05:12.998252 25421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b90891884a4db3bc8723b604cec5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:12.995769 25423 tablet_service.cc:1505] Processing CreateTablet for tablet 6e20042b566343a7acdbe06f4dcb0c86 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251210 21:05:13.003226 25423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20042b566343a7acdbe06f4dcb0c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.004904 25422 tablet_service.cc:1505] Processing CreateTablet for tablet bf7537ffe9f943b3a97e77adab5a1e23 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251210 21:05:13.005259 25422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7537ffe9f943b3a97e77adab5a1e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.006187 25428 tablet_service.cc:1505] Processing CreateTablet for tablet 063c8d31eded49ca9a780697c68d485d (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251210 21:05:13.006515 25428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063c8d31eded49ca9a780697c68d485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.007997 25504 tablet_service.cc:1505] Processing CreateTablet for tablet b13da827577e49569e304f0cbfb5e9df (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251210 21:05:13.008384 25504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b13da827577e49569e304f0cbfb5e9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.008427 25500 tablet_service.cc:1505] Processing CreateTablet for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251210 21:05:13.008874 25500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.009497 25503 tablet_service.cc:1505] Processing CreateTablet for tablet 544270c4ded24ee2ae32f6aa4ec505c2 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251210 21:05:13.009888 25503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544270c4ded24ee2ae32f6aa4ec505c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.011767 25502 tablet_service.cc:1505] Processing CreateTablet for tablet e7e5750d29e94bad943f054f32b40746 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251210 21:05:13.012202 25502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e5750d29e94bad943f054f32b40746. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.012120 25498 tablet_service.cc:1505] Processing CreateTablet for tablet b65abf7cc0484fcda919a741e23928a7 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251210 21:05:13.012475 25498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65abf7cc0484fcda919a741e23928a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.012805 25501 tablet_service.cc:1505] Processing CreateTablet for tablet 4810894f161642aba143677e92893e72 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251210 21:05:13.013168 25501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4810894f161642aba143677e92893e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.011767 25499 tablet_service.cc:1505] Processing CreateTablet for tablet 4f1698d0d3bd4765bc0897771cb0ceee (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251210 21:05:13.014894 25497 tablet_service.cc:1505] Processing CreateTablet for tablet 6e20042b566343a7acdbe06f4dcb0c86 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251210 21:05:13.014921 25495 tablet_service.cc:1505] Processing CreateTablet for tablet 05b90891884a4db3bc8723b604cec5ea (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251210 21:05:13.015259 25497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20042b566343a7acdbe06f4dcb0c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.017212 25496 tablet_service.cc:1505] Processing CreateTablet for tablet 681d997613474450ba6b9ee3de0f3f24 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251210 21:05:13.017598 25496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681d997613474450ba6b9ee3de0f3f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.019274 25495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b90891884a4db3bc8723b604cec5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.019608 25499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f1698d0d3bd4765bc0897771cb0ceee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.022822 25503 tablet_service.cc:1505] Processing CreateTablet for tablet bf7537ffe9f943b3a97e77adab5a1e23 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251210 21:05:13.024425 25503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7537ffe9f943b3a97e77adab5a1e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.028517 25504 tablet_service.cc:1505] Processing CreateTablet for tablet 063c8d31eded49ca9a780697c68d485d (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251210 21:05:13.029117 25504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063c8d31eded49ca9a780697c68d485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.033293 25560 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.036135 25560 tablet_bootstrap.cc:654] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.036402 25559 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.047384 25355 tablet_service.cc:1505] Processing CreateTablet for tablet b13da827577e49569e304f0cbfb5e9df (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251210 21:05:13.047966 25355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b13da827577e49569e304f0cbfb5e9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.048133 25353 tablet_service.cc:1505] Processing CreateTablet for tablet e7e5750d29e94bad943f054f32b40746 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251210 21:05:13.048573 25353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e5750d29e94bad943f054f32b40746. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.048806 25349 tablet_service.cc:1505] Processing CreateTablet for tablet b65abf7cc0484fcda919a741e23928a7 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251210 21:05:13.049237 25349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65abf7cc0484fcda919a741e23928a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.051894 25346 tablet_service.cc:1505] Processing CreateTablet for tablet 05b90891884a4db3bc8723b604cec5ea (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251210 21:05:13.051932 25347 tablet_service.cc:1505] Processing CreateTablet for tablet 681d997613474450ba6b9ee3de0f3f24 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251210 21:05:13.052366 25346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b90891884a4db3bc8723b604cec5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.052484 25348 tablet_service.cc:1505] Processing CreateTablet for tablet 6e20042b566343a7acdbe06f4dcb0c86 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251210 21:05:13.053400 25350 tablet_service.cc:1505] Processing CreateTablet for tablet 4f1698d0d3bd4765bc0897771cb0ceee (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251210 21:05:13.053961 25351 tablet_service.cc:1505] Processing CreateTablet for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251210 21:05:13.055640 25352 tablet_service.cc:1505] Processing CreateTablet for tablet 4810894f161642aba143677e92893e72 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251210 21:05:13.047420 25354 tablet_service.cc:1505] Processing CreateTablet for tablet 544270c4ded24ee2ae32f6aa4ec505c2 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251210 21:05:13.047570 25559 tablet_bootstrap.cc:654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.056136 25348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20042b566343a7acdbe06f4dcb0c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.057345 25350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f1698d0d3bd4765bc0897771cb0ceee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.057534 25351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.057549 25353 tablet_service.cc:1505] Processing CreateTablet for tablet bf7537ffe9f943b3a97e77adab5a1e23 (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251210 21:05:13.058998 25353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7537ffe9f943b3a97e77adab5a1e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.060002 25560 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.060215 25560 ts_tablet_manager.cc:1403] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20251210 21:05:13.064702 25347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681d997613474450ba6b9ee3de0f3f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.067683 25354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544270c4ded24ee2ae32f6aa4ec505c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.068547 25352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4810894f161642aba143677e92893e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.070823 25559 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.070960 25559 ts_tablet_manager.cc:1403] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.005s
I20251210 21:05:13.070943 25560 raft_consensus.cc:359] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.071135 25560 raft_consensus.cc:385] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.071192 25560 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.071389 25560 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.071655 25559 raft_consensus.cc:359] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.071767 25559 raft_consensus.cc:385] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.071799 25559 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.073400 25348 tablet_service.cc:1505] Processing CreateTablet for tablet 063c8d31eded49ca9a780697c68d485d (DEFAULT_TABLE table=test-workload [id=9cb6ac477b914209a88da6ddd266da43]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251210 21:05:13.073869 25348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063c8d31eded49ca9a780697c68d485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:13.071947 25559 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.083513 25563 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.084867 25560 ts_tablet_manager.cc:1434] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.025s user 0.003s sys 0.004s
I20251210 21:05:13.085246 25560 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.086194 25559 ts_tablet_manager.cc:1434] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20251210 21:05:13.087064 25560 tablet_bootstrap.cc:654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.090245 25559 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.092346 25559 tablet_bootstrap.cc:654] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.093400 25560 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.093577 25563 tablet_bootstrap.cc:654] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.093585 25560 ts_tablet_manager.cc:1403] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20251210 21:05:13.094341 25560 raft_consensus.cc:359] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.094586 25560 raft_consensus.cc:385] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.094693 25560 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.094936 25560 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.095410 25560 ts_tablet_manager.cc:1434] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:05:13.095724 25560 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.095822 25563 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.096000 25563 ts_tablet_manager.cc:1403] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20251210 21:05:13.096863 25563 raft_consensus.cc:359] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.097108 25563 raft_consensus.cc:385] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.097203 25560 tablet_bootstrap.cc:654] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.097213 25563 raft_consensus.cc:740] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.097556 25563 consensus_queue.cc:260] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.098322 25563 ts_tablet_manager.cc:1434] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:13.098595 25563 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.100323 25563 tablet_bootstrap.cc:654] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.100371 25560 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.100484 25560 ts_tablet_manager.cc:1403] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251210 21:05:13.095824 25559 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.101019 25560 raft_consensus.cc:359] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.101305 25560 raft_consensus.cc:385] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.101325 25559 ts_tablet_manager.cc:1403] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.001s
I20251210 21:05:13.101433 25560 raft_consensus.cc:740] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.101732 25560 consensus_queue.cc:260] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.102043 25559 raft_consensus.cc:359] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.102217 25559 raft_consensus.cc:385] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.102278 25559 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.102298 25560 ts_tablet_manager.cc:1434] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:13.102444 25559 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.102555 25560 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.103581 25559 ts_tablet_manager.cc:1434] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:05:13.103956 25559 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.103165 25563 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.104213 25563 ts_tablet_manager.cc:1403] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20251210 21:05:13.104939 25560 tablet_bootstrap.cc:654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.104929 25563 raft_consensus.cc:359] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.105196 25563 raft_consensus.cc:385] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.105324 25563 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.105626 25563 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.105981 25559 tablet_bootstrap.cc:654] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.106225 25563 ts_tablet_manager.cc:1434] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:05:13.106547 25563 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.108397 25563 tablet_bootstrap.cc:654] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.112844 25564 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.113034 25564 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.114205 25564 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.114563 25559 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.114622 25563 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.114758 25559 ts_tablet_manager.cc:1403] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20251210 21:05:13.114820 25563 ts_tablet_manager.cc:1403] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20251210 21:05:13.115571 25559 raft_consensus.cc:359] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.116405 25559 raft_consensus.cc:385] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.116834 25559 raft_consensus.cc:740] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.117146 25559 consensus_queue.cc:260] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.117730 25559 ts_tablet_manager.cc:1434] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251210 21:05:13.118089 25559 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.120102 25559 tablet_bootstrap.cc:654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.115604 25563 raft_consensus.cc:359] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.121304 25563 raft_consensus.cc:385] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.121399 25563 raft_consensus.cc:740] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.121706 25563 consensus_queue.cc:260] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.121906 25560 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.122066 25560 ts_tablet_manager.cc:1403] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.003s
I20251210 21:05:13.122357 25563 ts_tablet_manager.cc:1434] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251210 21:05:13.122635 25563 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.122952 25560 raft_consensus.cc:359] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.123207 25560 raft_consensus.cc:385] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.123337 25560 raft_consensus.cc:740] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.123678 25560 consensus_queue.cc:260] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.124502 25563 tablet_bootstrap.cc:654] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.124897 25560 ts_tablet_manager.cc:1434] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251210 21:05:13.125241 25560 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.130708 25560 tablet_bootstrap.cc:654] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.134234 25563 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.134421 25563 ts_tablet_manager.cc:1403] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20251210 21:05:13.134976 25560 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.135198 25560 ts_tablet_manager.cc:1403] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20251210 21:05:13.135805 25559 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.136070 25559 ts_tablet_manager.cc:1403] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.000s
I20251210 21:05:13.136065 25560 raft_consensus.cc:359] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.136303 25560 raft_consensus.cc:385] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.136435 25560 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.135217 25563 raft_consensus.cc:359] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.136781 25563 raft_consensus.cc:385] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.136873 25563 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.137180 25559 raft_consensus.cc:359] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.137423 25559 raft_consensus.cc:385] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.137454 25563 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.137571 25559 raft_consensus.cc:740] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.137132 25560 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.138181 25560 ts_tablet_manager.cc:1434] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251210 21:05:13.138784 25560 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.140827 25560 tablet_bootstrap.cc:654] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.138371 25559 consensus_queue.cc:260] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.142268 25559 ts_tablet_manager.cc:1434] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251210 21:05:13.142297 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.142565 25559 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.142664 25563 ts_tablet_manager.cc:1434] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251210 21:05:13.144369 25559 tablet_bootstrap.cc:654] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.144425 25440 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.144716 25440 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
W20251210 21:05:13.143371 25479 leader_election.cc:343] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.145629 25477 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.145947 25564 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.146070 25564 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.146162 25564 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.149256 25563 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.149338 25564 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.150727 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
W20251210 21:05:13.151312 25479 leader_election.cc:343] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.151826 25563 tablet_bootstrap.cc:654] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.152171 25440 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:13.152362 25440 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.153980 25560 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.154124 25560 ts_tablet_manager.cc:1403] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20251210 21:05:13.154875 25560 raft_consensus.cc:359] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.155041 25560 raft_consensus.cc:385] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.155115 25560 raft_consensus.cc:740] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.155165 25440 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.155337 25560 consensus_queue.cc:260] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.155701 25477 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.156879 25564 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.157053 25559 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.157404 25559 ts_tablet_manager.cc:1403] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20251210 21:05:13.158272 25559 raft_consensus.cc:359] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.158535 25559 raft_consensus.cc:385] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.158663 25559 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.159021 25559 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.159658 25559 ts_tablet_manager.cc:1434] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:05:13.159940 25564 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.160020 25559 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.160424 25564 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.160907 25564 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.162092 25559 tablet_bootstrap.cc:654] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.177210 25563 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.177487 25563 ts_tablet_manager.cc:1403] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.015s
I20251210 21:05:13.178470 25262 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:13.179142 25560 ts_tablet_manager.cc:1434] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.025s user 0.006s sys 0.014s
I20251210 21:05:13.179497 25560 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.178442 25563 raft_consensus.cc:359] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.179765 25563 raft_consensus.cc:385] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.179885 25563 raft_consensus.cc:740] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.179942 25559 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.180162 25559 ts_tablet_manager.cc:1403] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.012s
I20251210 21:05:13.180184 25563 consensus_queue.cc:260] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.180838 25563 ts_tablet_manager.cc:1434] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251210 21:05:13.181089 25559 raft_consensus.cc:359] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.181226 25563 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.181339 25559 raft_consensus.cc:385] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.181512 25559 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.181686 25560 tablet_bootstrap.cc:654] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.181813 25559 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.182374 25559 ts_tablet_manager.cc:1434] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:05:13.183878 25563 tablet_bootstrap.cc:654] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.184404 25559 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.185643 25560 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.185845 25560 ts_tablet_manager.cc:1403] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251210 21:05:13.186028 25559 tablet_bootstrap.cc:654] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.186664 25560 raft_consensus.cc:359] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.186856 25560 raft_consensus.cc:385] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.186949 25560 raft_consensus.cc:740] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.187198 25560 consensus_queue.cc:260] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.187716 25560 ts_tablet_manager.cc:1434] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:13.188027 25560 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.189834 25560 tablet_bootstrap.cc:654] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.191880 25563 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.192095 25563 ts_tablet_manager.cc:1403] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20251210 21:05:13.192898 25563 raft_consensus.cc:359] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.193054 25563 raft_consensus.cc:385] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.193137 25563 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.193213 25560 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.193415 25560 ts_tablet_manager.cc:1403] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20251210 21:05:13.193358 25563 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.193951 25563 ts_tablet_manager.cc:1434] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:05:13.194170 25560 raft_consensus.cc:359] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.194293 25563 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.194396 25560 raft_consensus.cc:385] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.194517 25560 raft_consensus.cc:740] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.194748 25560 consensus_queue.cc:260] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.195302 25560 ts_tablet_manager.cc:1434] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:05:13.195643 25560 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.196097 25563 tablet_bootstrap.cc:654] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.197463 25560 tablet_bootstrap.cc:654] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.194315 25559 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.197816 25569 raft_consensus.cc:493] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.197993 25559 ts_tablet_manager.cc:1403] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.005s
I20251210 21:05:13.197990 25569 raft_consensus.cc:515] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.198803 25559 raft_consensus.cc:359] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.198980 25559 raft_consensus.cc:385] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.199056 25559 raft_consensus.cc:740] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.199093 25569 leader_election.cc:290] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.199327 25559 consensus_queue.cc:260] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.199910 25559 ts_tablet_manager.cc:1434] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:13.200196 25559 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.202100 25559 tablet_bootstrap.cc:654] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.206233 25583 raft_consensus.cc:493] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.206398 25583 raft_consensus.cc:515] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.207223 25583 leader_election.cc:290] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.207921 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.208446 25563 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.208602 25563 ts_tablet_manager.cc:1403] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20251210 21:05:13.208657 25440 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.208863 25440 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.209422 25477 leader_election.cc:304] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.209458 25563 raft_consensus.cc:359] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.209611 25563 raft_consensus.cc:385] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.209714 25563 raft_consensus.cc:740] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.209944 25563 consensus_queue.cc:260] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:13.210096 25479 leader_election.cc:343] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.210480 25563 ts_tablet_manager.cc:1434] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:05:13.210783 25563 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.210482 25583 raft_consensus.cc:2804] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.211421 25583 raft_consensus.cc:493] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.211540 25583 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.212766 25563 tablet_bootstrap.cc:654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.214485 25583 raft_consensus.cc:515] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.214859 25560 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.215035 25560 ts_tablet_manager.cc:1403] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.008s
I20251210 21:05:13.215797 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.215840 25560 raft_consensus.cc:359] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.216040 25560 raft_consensus.cc:385] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.216115 25560 raft_consensus.cc:740] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.216385 25560 consensus_queue.cc:260] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:13.216506 25479 leader_election.cc:343] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.217041 25560 ts_tablet_manager.cc:1434] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:13.217494 25560 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.219601 25560 tablet_bootstrap.cc:654] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.222376 25440 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:13.222606 25440 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.225850 25440 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.226615 25477 leader_election.cc:304] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.227065 25564 raft_consensus.cc:2804] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.227216 25564 raft_consensus.cc:697] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.227530 25564 consensus_queue.cc:237] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.227933 25560 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.228119 25560 ts_tablet_manager.cc:1403] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20251210 21:05:13.229050 25584 raft_consensus.cc:493] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.229060 25560 raft_consensus.cc:359] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.229207 25584 raft_consensus.cc:515] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.230142 25584 leader_election.cc:290] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.229331 25560 raft_consensus.cc:385] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.230646 25560 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.230924 25560 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.231705 25560 ts_tablet_manager.cc:1434] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251210 21:05:13.231778 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "063c8d31eded49ca9a780697c68d485d" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.232091 25560 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
W20251210 21:05:13.232425 25479 leader_election.cc:343] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.234225 25560 tablet_bootstrap.cc:654] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.235491 25263 catalog_manager.cc:5654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:13.240465 25440 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "063c8d31eded49ca9a780697c68d485d" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.240833 25440 raft_consensus.cc:2468] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.241338 25477 leader_election.cc:304] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.241847 25564 raft_consensus.cc:2804] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.241947 25564 raft_consensus.cc:493] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.242061 25564 raft_consensus.cc:3060] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.242728 25560 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.242918 25560 ts_tablet_manager.cc:1403] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20251210 21:05:13.243872 25560 raft_consensus.cc:359] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.244095 25560 raft_consensus.cc:385] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.244172 25560 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.244448 25560 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.245359 25564 raft_consensus.cc:515] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.246274 25560 ts_tablet_manager.cc:1434] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251210 21:05:13.246668 25560 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Bootstrap starting.
I20251210 21:05:13.246670 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "063c8d31eded49ca9a780697c68d485d" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
W20251210 21:05:13.247376 25479 leader_election.cc:343] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.248310 25564 leader_election.cc:290] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.248436 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "063c8d31eded49ca9a780697c68d485d" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:13.248708 25439 raft_consensus.cc:3060] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.248980 25560 tablet_bootstrap.cc:654] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.252000 25439 raft_consensus.cc:2468] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.252568 25477 leader_election.cc:304] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.252665 25583 leader_election.cc:290] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.256690 25565 raft_consensus.cc:493] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.256877 25565 raft_consensus.cc:515] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.257942 25565 leader_election.cc:290] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.258245 25560 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: No bootstrap required, opened a new log
I20251210 21:05:13.258456 25560 ts_tablet_manager.cc:1403] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20251210 21:05:13.259395 25560 raft_consensus.cc:359] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.259735 25560 raft_consensus.cc:385] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.259866 25560 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.260180 25560 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.260530 25584 raft_consensus.cc:2804] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.266942 25560 ts_tablet_manager.cc:1434] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20251210 21:05:13.304816 25564 raft_consensus.cc:493] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.305124 25564 raft_consensus.cc:515] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.306229 25564 leader_election.cc:290] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.307277 25569 raft_consensus.cc:493] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.307509 25569 raft_consensus.cc:515] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.308189 25569 leader_election.cc:290] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.308857 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.309131 25365 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.309576 25479 leader_election.cc:304] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.310145 25564 raft_consensus.cc:2804] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.310250 25564 raft_consensus.cc:493] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.310338 25564 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.313503 25564 raft_consensus.cc:515] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.314790 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.315001 25365 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.318264 25365 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.318821 25479 leader_election.cc:304] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.319265 25583 raft_consensus.cc:2804] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.319483 25583 raft_consensus.cc:697] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.319851 25583 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.321547 25564 leader_election.cc:290] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.330741 25564 raft_consensus.cc:493] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.331043 25564 raft_consensus.cc:515] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.332360 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.332662 25365 raft_consensus.cc:2468] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.333225 25479 leader_election.cc:304] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.333920 25564 leader_election.cc:290] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.334156 25564 raft_consensus.cc:2804] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.334288 25564 raft_consensus.cc:493] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.334602 25564 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.339284 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.339579 25439 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.340642 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:13.340943 25439 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.344319 25439 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.345434 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
W20251210 21:05:13.346999 25477 leader_election.cc:343] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.349882 25564 raft_consensus.cc:515] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.355314 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.355652 25365 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.359134 25365 raft_consensus.cc:2468] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.359923 25479 leader_election.cc:304] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.362309 25584 raft_consensus.cc:697] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.362859 25563 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.362812 25584 consensus_queue.cc:237] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.363157 25563 ts_tablet_manager.cc:1403] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.152s user 0.057s sys 0.089s
I20251210 21:05:13.362393 25602 raft_consensus.cc:2804] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.364217 25563 raft_consensus.cc:359] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.364513 25563 raft_consensus.cc:385] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.364655 25563 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.364950 25563 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.365605 25563 ts_tablet_manager.cc:1434] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:05:13.365986 25563 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.367929 25563 tablet_bootstrap.cc:654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.372660 25602 raft_consensus.cc:697] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.373157 25602 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.378760 25263 catalog_manager.cc:5654] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:13.382985 25602 raft_consensus.cc:493] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.383273 25602 raft_consensus.cc:515] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.383750 25564 leader_election.cc:290] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.384920 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.385238 25365 raft_consensus.cc:2468] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.385092 25263 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:13.385696 25479 leader_election.cc:304] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.385854 25263 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:13.386092 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
W20251210 21:05:13.386538 25477 leader_election.cc:343] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.387420 25602 leader_election.cc:290] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.387367 25514 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.387635 25514 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 0.
I20251210 21:05:13.387710 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
W20251210 21:05:13.388130 25477 leader_election.cc:343] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.388532 25403 leader_election.cc:304] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.389263 25565 raft_consensus.cc:2804] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.389387 25565 raft_consensus.cc:493] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.389494 25565 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.389897 25559 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.390055 25559 ts_tablet_manager.cc:1403] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.190s user 0.062s sys 0.116s
I20251210 21:05:13.390241 25563 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.390384 25563 ts_tablet_manager.cc:1403] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.024s user 0.013s sys 0.006s
I20251210 21:05:13.390657 25559 raft_consensus.cc:359] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.390802 25559 raft_consensus.cc:385] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.390859 25559 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.391064 25559 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.391145 25563 raft_consensus.cc:359] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.391306 25563 raft_consensus.cc:385] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.391417 25563 raft_consensus.cc:740] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.391687 25563 consensus_queue.cc:260] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.392210 25563 ts_tablet_manager.cc:1434] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:13.392519 25563 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.392867 25565 raft_consensus.cc:515] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.394088 25514 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:13.394272 25514 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.394434 25563 tablet_bootstrap.cc:654] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.396237 25565 leader_election.cc:290] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.399709 25514 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 1.
I20251210 21:05:13.400245 25403 leader_election.cc:304] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.400584 25565 raft_consensus.cc:2804] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.400735 25565 raft_consensus.cc:697] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:13.401386 25602 raft_consensus.cc:493] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.401533 25602 raft_consensus.cc:515] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.386052 25564 raft_consensus.cc:2804] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.402215 25602 leader_election.cc:290] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:13.402758 25564 raft_consensus.cc:493] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.402943 25564 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.403353 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.403636 25439 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.404002 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.404188 25365 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.405066 25565 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.408095 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.408239 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.408454 25365 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 0.
I20251210 21:05:13.408906 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.408834 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.409263 25364 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
W20251210 21:05:13.409868 25328 leader_election.cc:343] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): Illegal state: must be running to vote when last-logged opid is not known
W20251210 21:05:13.410215 25328 leader_election.cc:343] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.410465 25477 leader_election.cc:304] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.411341 25602 raft_consensus.cc:2804] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.411554 25602 raft_consensus.cc:493] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.411662 25602 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.412839 25364 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 1.
I20251210 21:05:13.414568 25602 raft_consensus.cc:515] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.415771 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:13.415997 25439 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.418849 25439 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.419315 25477 leader_election.cc:304] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.419674 25584 raft_consensus.cc:2804] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.419816 25584 raft_consensus.cc:697] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.420107 25584 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:13.422205 25602 leader_election.cc:290] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:13.422307 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.422631 25364 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.423661 25261 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:13.425616 25514 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.425657 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.425855 25514 raft_consensus.cc:2468] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 0.
I20251210 21:05:13.426363 25328 leader_election.cc:304] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:13.427098 25569 raft_consensus.cc:2804] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.427204 25569 raft_consensus.cc:493] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.427309 25569 raft_consensus.cc:3060] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.428954 25364 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.430155 25569 raft_consensus.cc:515] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.430855 25569 leader_election.cc:290] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.431753 25559 ts_tablet_manager.cc:1434] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.042s user 0.017s sys 0.025s
I20251210 21:05:13.432112 25559 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.432353 25563 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.432551 25563 ts_tablet_manager.cc:1403] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.040s user 0.012s sys 0.026s
I20251210 21:05:13.433120 25328 leader_election.cc:304] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89; no voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:13.433163 25261 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 1 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:13.433467 25569 raft_consensus.cc:2749] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:13.433494 25563 raft_consensus.cc:359] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.433673 25563 raft_consensus.cc:385] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.433761 25563 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.433995 25559 tablet_bootstrap.cc:654] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.434047 25563 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.434263 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:13.434453 25513 raft_consensus.cc:3060] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.435562 25563 ts_tablet_manager.cc:1434] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251210 21:05:13.435896 25563 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.437496 25513 raft_consensus.cc:2468] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 1.
I20251210 21:05:13.437592 25563 tablet_bootstrap.cc:654] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.438184 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
W20251210 21:05:13.438673 25328 leader_election.cc:343] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.438992 25328 leader_election.cc:304] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:13.440102 25569 raft_consensus.cc:2804] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.441102 25569 raft_consensus.cc:697] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:13.441416 25569 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.444214 25559 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.444428 25559 ts_tablet_manager.cc:1403] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20251210 21:05:13.444351 25261 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:13.445222 25559 raft_consensus.cc:359] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.445394 25559 raft_consensus.cc:385] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.445434 25564 raft_consensus.cc:515] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.445581 25559 raft_consensus.cc:740] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.445833 25559 consensus_queue.cc:260] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.446367 25559 ts_tablet_manager.cc:1434] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:05:13.446449 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.446602 25364 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.446655 25559 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.448414 25559 tablet_bootstrap.cc:654] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.449415 25364 raft_consensus.cc:2468] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.449841 25569 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.449998 25569 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.450608 25569 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.450847 25479 leader_election.cc:304] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.451150 25602 raft_consensus.cc:2804] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.451277 25602 raft_consensus.cc:697] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.451606 25602 consensus_queue.cc:237] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.452450 25563 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.452667 25563 ts_tablet_manager.cc:1403] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.002s
I20251210 21:05:13.452806 25559 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.453034 25559 ts_tablet_manager.cc:1403] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.006s
I20251210 21:05:13.453372 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.453475 25563 raft_consensus.cc:359] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.453606 25563 raft_consensus.cc:385] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.453672 25563 raft_consensus.cc:740] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.453821 25559 raft_consensus.cc:359] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.453864 25563 consensus_queue.cc:260] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.454020 25559 raft_consensus.cc:385] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.454128 25559 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.454152 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.454418 25563 ts_tablet_manager.cc:1434] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:05:13.454444 25439 raft_consensus.cc:2393] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 128e4ab89fe449e8b5fdc68590b7ac89 in current term 1: Already voted for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in this term.
I20251210 21:05:13.454388 25559 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.454775 25563 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: Bootstrap starting.
I20251210 21:05:13.455356 25328 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89; no voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:13.456303 25569 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.456631 25563 tablet_bootstrap.cc:654] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.457027 25262 catalog_manager.cc:5654] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:13.455583 25559 ts_tablet_manager.cc:1434] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:05:13.458210 25559 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.459164 25569 raft_consensus.cc:2749] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:13.459906 25559 tablet_bootstrap.cc:654] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.460439 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
W20251210 21:05:13.460947 25477 leader_election.cc:343] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.461203 25564 leader_election.cc:290] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.463229 25559 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.463390 25559 ts_tablet_manager.cc:1403] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251210 21:05:13.464304 25559 raft_consensus.cc:359] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.464442 25559 raft_consensus.cc:385] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.464506 25559 raft_consensus.cc:740] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.464661 25559 consensus_queue.cc:260] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.465116 25559 ts_tablet_manager.cc:1434] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:05:13.465289 25564 raft_consensus.cc:493] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.465420 25559 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: Bootstrap starting.
I20251210 21:05:13.465427 25564 raft_consensus.cc:515] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.466187 25564 leader_election.cc:290] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.466529 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.466773 25364 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.466813 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.466995 25439 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 0.
I20251210 21:05:13.467135 25610 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.467227 25559 tablet_bootstrap.cc:654] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:13.467264 25610 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.467353 25479 leader_election.cc:304] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.467680 25564 raft_consensus.cc:2804] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.467792 25564 raft_consensus.cc:493] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.467914 25564 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.468041 25610 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.468662 25563 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: No bootstrap required, opened a new log
I20251210 21:05:13.468861 25563 ts_tablet_manager.cc:1403] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20251210 21:05:13.469152 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.469602 25563 raft_consensus.cc:359] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.469844 25563 raft_consensus.cc:385] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.469930 25563 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Initialized, Role: FOLLOWER
W20251210 21:05:13.469625 25405 leader_election.cc:343] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:13.470163 25563 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.470841 25563 ts_tablet_manager.cc:1434] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:05:13.470948 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.471118 25513 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 0.
I20251210 21:05:13.471506 25403 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters: 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.471926 25610 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.471815 25564 raft_consensus.cc:515] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.472030 25610 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.472131 25610 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.472865 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.473026 25364 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.474879 25610 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.475595 25364 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.475796 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.475829 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:13.475996 25479 leader_election.cc:304] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.476274 25365 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.476320 25602 raft_consensus.cc:2804] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.476467 25602 raft_consensus.cc:697] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.476724 25602 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.479218 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:13.479418 25439 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.479318 25261 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:13.476289 25513 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.481551 25439 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 1.
I20251210 21:05:13.479250 25365 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 1.
I20251210 21:05:13.481921 25564 leader_election.cc:290] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:13.482434 25513 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 1.
I20251210 21:05:13.476300 25610 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.482762 25403 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:13.483206 25559 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: No bootstrap required, opened a new log
I20251210 21:05:13.483186 25610 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.483403 25559 ts_tablet_manager.cc:1403] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20251210 21:05:13.483400 25610 raft_consensus.cc:697] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:13.483717 25610 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.484216 25559 raft_consensus.cc:359] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.484478 25559 raft_consensus.cc:385] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:13.484573 25559 raft_consensus.cc:740] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Initialized, Role: FOLLOWER
I20251210 21:05:13.484813 25559 consensus_queue.cc:260] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.485333 25559 ts_tablet_manager.cc:1434] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:05:13.487120 25261 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 1 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:13.553452 25610 raft_consensus.cc:493] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.553663 25610 raft_consensus.cc:515] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.554261 25610 leader_election.cc:290] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.554837 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.554837 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.555141 25365 raft_consensus.cc:2393] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4af27de48bec4590bfcd8d71056814eb in current term 1: Already voted for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in this term.
I20251210 21:05:13.555508 25405 leader_election.cc:304] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb; no voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:13.555761 25610 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.557632 25610 raft_consensus.cc:2749] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:13.583808 25569 raft_consensus.cc:493] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.584000 25569 raft_consensus.cc:515] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.584599 25569 leader_election.cc:290] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.585094 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.585119 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.585376 25439 raft_consensus.cc:2393] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 128e4ab89fe449e8b5fdc68590b7ac89 in current term 1: Already voted for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in this term.
I20251210 21:05:13.585721 25328 leader_election.cc:304] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89; no voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:13.586019 25569 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.587824 25569 raft_consensus.cc:2749] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:13.592686 25569 raft_consensus.cc:493] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.592861 25569 raft_consensus.cc:515] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.593418 25569 leader_election.cc:290] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.593870 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.593958 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.594085 25439 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 0.
I20251210 21:05:13.594156 25513 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 0.
I20251210 21:05:13.594444 25328 leader_election.cc:304] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:13.594702 25569 raft_consensus.cc:2804] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:13.594779 25569 raft_consensus.cc:493] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:13.594872 25569 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.596712 25569 raft_consensus.cc:515] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.597199 25569 leader_election.cc:290] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.597612 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:13.597605 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:13.597803 25513 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.597832 25439 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.600528 25439 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 1.
I20251210 21:05:13.600533 25513 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 1.
I20251210 21:05:13.600910 25328 leader_election.cc:304] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:13.601120 25569 raft_consensus.cc:2804] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:13.601260 25569 raft_consensus.cc:697] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:13.601519 25569 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.604009 25261 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:13.615758 25610 raft_consensus.cc:493] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.615927 25610 raft_consensus.cc:515] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.616520 25610 leader_election.cc:290] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.616989 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.617028 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.617262 25513 raft_consensus.cc:2393] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4af27de48bec4590bfcd8d71056814eb in current term 1: Already voted for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in this term.
I20251210 21:05:13.617615 25403 leader_election.cc:304] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb; no voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:13.617851 25610 raft_consensus.cc:3060] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.619626 25610 raft_consensus.cc:2749] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:13.638201 25602 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.639106 25602 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.644579 25583 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.646324 25583 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.681825 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:13.690565 25513 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:13.690832 25513 raft_consensus.cc:606] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.691921 25513 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:13.692107 25513 raft_consensus.cc:606] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:13.699048 25513 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:13.699215 25513 raft_consensus.cc:606] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:13.700040 25513 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:13.700168 25513 raft_consensus.cc:606] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:13.700470 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20251210 21:05:13.700645 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:13.709570 25583 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.713382 25564 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.716483 25584 raft_consensus.cc:993] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:13.716681 25584 raft_consensus.cc:1081] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:13.717226 25439 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43842
I20251210 21:05:13.717489 25439 raft_consensus.cc:493] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:13.717649 25439 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:13.721343 25439 raft_consensus.cc:515] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.721884 25439 leader_election.cc:290] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.722324 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:13.722337 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:13.722496 25513 raft_consensus.cc:3055] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:13.722536 25365 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:13.722558 25513 raft_consensus.cc:740] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:13.722769 25513 consensus_queue.cc:260] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.723194 25513 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:13.725327 25365 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:13.725658 25513 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:13.725674 25405 leader_election.cc:304] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:13.726007 25610 raft_consensus.cc:2804] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:13.726118 25610 raft_consensus.cc:697] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:13.726332 25610 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.729526 25263 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 1 to 2, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:13.730578 25569 raft_consensus.cc:493] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.730695 25569 raft_consensus.cc:515] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.731245 25569 leader_election.cc:290] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.734133 25439 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "063c8d31eded49ca9a780697c68d485d" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af27de48bec4590bfcd8d71056814eb" is_pre_election: true
I20251210 21:05:13.734344 25439 raft_consensus.cc:2393] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 128e4ab89fe449e8b5fdc68590b7ac89 in current term 1: Already voted for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in this term.
I20251210 21:05:13.739387 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "063c8d31eded49ca9a780697c68d485d" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.739918 25328 leader_election.cc:304] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89; no voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:13.740379 25569 raft_consensus.cc:3060] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.745215 25569 raft_consensus.cc:2749] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:13.759334 25565 raft_consensus.cc:493] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:13.759514 25565 raft_consensus.cc:515] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:13.760073 25565 leader_election.cc:290] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:13.760453 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" is_pre_election: true
I20251210 21:05:13.760507 25513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" is_pre_election: true
I20251210 21:05:13.760700 25365 raft_consensus.cc:2393] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4af27de48bec4590bfcd8d71056814eb in current term 1: Already voted for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in this term.
I20251210 21:05:13.761036 25405 leader_election.cc:304] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb; no voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:13.761257 25565 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:13.764020 25565 raft_consensus.cc:2749] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:13.767189 25584 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.768141 25584 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.802989 25513 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.803203 25439 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.803584 25514 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.802989 25365 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:13.804514 25569 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.805703 25616 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.806129 25616 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.806478 25514 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.805678 25610 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20251210 21:05:13.806927 25565 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.807022 25513 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:13.807751 25565 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251210 21:05:13.820246 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.824402 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.833935 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.834703 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:13.836396 25365 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251210 21:05:13.840840 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.841894 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.843037 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.843732 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.844393 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:13.844884 25564 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.845973 25440 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.846235 25610 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.846470 25647 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.849428 25364 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.853346 25510 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.865072 25610 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:05:13.866848 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.867810 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.868582 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.869148 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.869674 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.870199 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:13.870720 25610 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.878408 25359 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251210 21:05:13.885743 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.885756 25489 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:13.888979 25438 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:13.889053 25362 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251210 21:05:13.892652 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:13.891157 25607 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20251210 21:05:13.902558 25584 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:05:13.903954 25486 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.904989 25492 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.906163 25489 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.904536 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.909981 25489 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.904758 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.910646 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.910399 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.922742 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.931470 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.935894 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:13.960732 25584 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:13.960875 25602 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20251210 21:05:13.969316 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.972995 25650 log.cc:927] Time spent T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Append to log took a long time: real 0.084s user 0.000s sys 0.000s
W20251210 21:05:13.986184 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.988418 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:13.989446 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.066951 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.072696 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.073304 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.075114 25584 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:05:14.079644 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.088879 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.093159 25584 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:14.093971 25584 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:05:14.135524 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.136591 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.137400 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.138253 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.141705 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.146682 25583 raft_consensus.cc:993] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:14.146812 25583 raft_consensus.cc:1081] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
W20251210 21:05:14.168285 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.184566 25490 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.190434 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.190940 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.194979 25491 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.196074 25602 raft_consensus.cc:993] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:14.196188 25602 raft_consensus.cc:1081] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
W20251210 21:05:14.204567 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.209987 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.215649 25646 mvcc.cc:204] Tried to move back new op lower bound from 7231081324078411776 to 7231081322171215872. Current Snapshot: MvccSnapshot[applied={T|T < 7231081324022628352}]
I20251210 21:05:14.218031 25438 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43842
I20251210 21:05:14.218230 25438 raft_consensus.cc:493] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:14.218319 25438 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:14.223804 25363 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39018
I20251210 21:05:14.224011 25363 raft_consensus.cc:493] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:14.224104 25363 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
W20251210 21:05:14.224848 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.260970 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.265475 25438 raft_consensus.cc:515] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.266018 25438 leader_election.cc:290] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:14.270238 25361 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:14.270409 25361 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:14.271209 25584 raft_consensus.cc:993] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:14.271329 25584 raft_consensus.cc:1081] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
W20251210 21:05:14.275529 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.276204 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.276734 25510 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:14.276882 25510 raft_consensus.cc:3055] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:14.276947 25510 raft_consensus.cc:740] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
W20251210 21:05:14.277647 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.279248 25510 consensus_queue.cc:260] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.279562 25510 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:14.280333 25365 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39018
I20251210 21:05:14.280512 25365 raft_consensus.cc:493] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:14.280601 25365 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
W20251210 21:05:14.285022 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
W20251210 21:05:14.306128 25493 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.306128 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.307265 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
W20251210 21:05:14.309036 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.318265 25363 raft_consensus.cc:515] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.320358 25363 leader_election.cc:290] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:14.321266 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:14.321305 25512 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:14.321434 25435 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:14.321463 25512 raft_consensus.cc:3055] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:14.321511 25512 raft_consensus.cc:740] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:14.321702 25512 consensus_queue.cc:260] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.321977 25512 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:14.323566 25361 raft_consensus.cc:2468] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:14.323966 25405 leader_election.cc:304] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:14.324247 25686 raft_consensus.cc:2804] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:14.324398 25686 raft_consensus.cc:697] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
W20251210 21:05:14.324226 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
I20251210 21:05:14.324597 25686 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:14.329020 25494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
W20251210 21:05:14.329020 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.329797 25512 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
W20251210 21:05:14.330638 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
W20251210 21:05:14.331117 25345 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39036: Illegal state: replica 128e4ab89fe449e8b5fdc68590b7ac89 is not leader of this config: current role FOLLOWER
W20251210 21:05:14.331982 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Service unavailable: leader transfer in progress
I20251210 21:05:14.332929 25328 leader_election.cc:304] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:14.333271 25616 raft_consensus.cc:2804] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:14.333228 25365 raft_consensus.cc:515] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.333380 25616 raft_consensus.cc:697] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:14.333621 25616 consensus_queue.cc:237] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.333791 25510 raft_consensus.cc:2468] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:14.333791 25435 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:14.333988 25365 leader_election.cc:290] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:14.334237 25258 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 1 to 2, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
W20251210 21:05:14.336103 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
W20251210 21:05:14.337318 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
W20251210 21:05:14.337586 25345 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39036: Illegal state: replica 128e4ab89fe449e8b5fdc68590b7ac89 is not leader of this config: current role FOLLOWER
I20251210 21:05:14.338128 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:14.338315 25510 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:14.338328 25435 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:14.338511 25510 raft_consensus.cc:3055] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:14.338595 25510 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:14.338876 25510 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.339212 25510 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
W20251210 21:05:14.341140 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
W20251210 21:05:14.342885 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
I20251210 21:05:14.343390 25365 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:14.344149 25435 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:14.344405 25512 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:14.344436 25610 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20251210 21:05:14.344565 25328 leader_election.cc:304] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:14.344794 25647 raft_consensus.cc:2804] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:14.344957 25647 raft_consensus.cc:697] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
W20251210 21:05:14.344983 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
I20251210 21:05:14.345258 25647 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:14.345770 25610 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:14.345659 25510 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
W20251210 21:05:14.349388 25487 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
W20251210 21:05:14.350001 25485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47122: Illegal state: replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config: current role FOLLOWER
I20251210 21:05:14.350916 25512 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:14.351476 25647 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:14.365592 25257 catalog_manager.cc:5654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 1 to 2, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 2 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:14.366346 25257 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 1 to 2, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 2 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:14.366585 25435 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:14.370136 25616 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:14.371137 25435 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:14.371317 25510 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:05:14.371649 25616 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:14.372056 25616 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:15.333133 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:15.336297 25704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:15.337550 25705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:15.340987 25707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:15.341168 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:15.341475 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:15.341543 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:15.341600 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400715341567 us; error 0 us; skew 500 ppm
I20251210 21:05:15.341746 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:15.342981 22013 webserver.cc:492] Webserver started at http://127.21.127.68:39301/ using document root <none> and password file <none>
I20251210 21:05:15.343211 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:15.343305 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:15.343475 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:15.344120 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
format_stamp: "Formatted at 2025-12-10 21:05:15 on dist-test-slave-9pq8"
I20251210 21:05:15.346691 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:05:15.348361 25712 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.348726 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:15.348831 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
format_stamp: "Formatted at 2025-12-10 21:05:15 on dist-test-slave-9pq8"
I20251210 21:05:15.348968 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:15.370507 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:15.371117 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:15.371750 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:15.372766 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:15.372830 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.372915 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:15.372977 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.389194 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:37845
I20251210 21:05:15.389226 25774 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:37845 every 8 connection(s)
I20251210 21:05:15.390935 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:15.396421 25779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:15.397675 25780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:15.398339 25782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:15.403224 25775 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44841
I20251210 21:05:15.403364 25775 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:15.403787 25775 heartbeater.cc:507] Master 127.21.127.126:44841 requested a full tablet report, sending...
I20251210 21:05:15.404618 25258 ts_manager.cc:194] Registered new tserver with Master: dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845)
I20251210 21:05:15.405576 25258 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48818
I20251210 21:05:15.405627 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:15.409417 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:15.409507 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:15.409539 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400715409536 us; error 0 us; skew 500 ppm
I20251210 21:05:15.409667 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:15.410930 22013 webserver.cc:492] Webserver started at http://127.21.127.69:43837/ using document root <none> and password file <none>
I20251210 21:05:15.411139 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:15.411208 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:15.411341 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:15.411960 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-4-root/instance:
uuid: "a6abef32d32c46478bc3f2b9aa589efb"
format_stamp: "Formatted at 2025-12-10 21:05:15 on dist-test-slave-9pq8"
I20251210 21:05:15.414266 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251210 21:05:15.415884 25787 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.416247 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:05:15.416358 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-4-root
uuid: "a6abef32d32c46478bc3f2b9aa589efb"
format_stamp: "Formatted at 2025-12-10 21:05:15 on dist-test-slave-9pq8"
I20251210 21:05:15.416455 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-4-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-4-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-4-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:15.427695 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:15.428231 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:15.428742 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:15.429524 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:15.429581 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.429653 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:15.429687 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.441461 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.69:45341
I20251210 21:05:15.441514 25849 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.69:45341 every 8 connection(s)
I20251210 21:05:15.443231 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:15.445880 25853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:15.446302 25854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:15.447618 25856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:15.448194 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:15.448515 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:15.448568 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:15.448598 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400715448594 us; error 0 us; skew 500 ppm
I20251210 21:05:15.448781 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:15.449630 25850 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44841
I20251210 21:05:15.449755 25850 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:15.450208 25850 heartbeater.cc:507] Master 127.21.127.126:44841 requested a full tablet report, sending...
I20251210 21:05:15.450235 22013 webserver.cc:492] Webserver started at http://127.21.127.70:35093/ using document root <none> and password file <none>
I20251210 21:05:15.450417 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:15.450484 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:15.450626 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:15.451076 25258 ts_manager.cc:194] Registered new tserver with Master: a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:15.451311 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-5-root/instance:
uuid: "90c04d3cdd4b4cb18b2a265365b1b094"
format_stamp: "Formatted at 2025-12-10 21:05:15 on dist-test-slave-9pq8"
I20251210 21:05:15.452075 25258 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48820
I20251210 21:05:15.453995 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:15.455402 25861 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.455720 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:15.455827 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-5-root
uuid: "90c04d3cdd4b4cb18b2a265365b1b094"
format_stamp: "Formatted at 2025-12-10 21:05:15 on dist-test-slave-9pq8"
I20251210 21:05:15.455910 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-5-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-5-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-5-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:15.497053 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:15.497547 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:15.498042 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:15.498893 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:15.498950 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.498997 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:15.499025 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:15.513304 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.70:38549
I20251210 21:05:15.513342 25923 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.70:38549 every 8 connection(s)
I20251210 21:05:15.520237 25924 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44841
I20251210 21:05:15.520377 25924 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:15.520736 25924 heartbeater.cc:507] Master 127.21.127.126:44841 requested a full tablet report, sending...
I20251210 21:05:15.521541 25258 ts_manager.cc:194] Registered new tserver with Master: 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549)
I20251210 21:05:15.522372 25258 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48826
I20251210 21:05:15.701027 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:15.703756 25509 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:15.703974 25509 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:15.704443 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:15.704613 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:15.726161 25602 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:15.726320 25602 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:15.727093 25364 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39018
I20251210 21:05:15.727299 25364 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:15.727398 25364 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:15.730353 25364 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:15.730957 25364 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:15.731539 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 25 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:15.731725 25509 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:15.731957 25509 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:15.732049 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 25 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:15.732234 25435 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:15.732270 25509 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 25, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:15.732609 25509 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:15.734133 25436 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:15.734933 25435 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:15.735378 25328 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:15.735647 25703 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:15.735754 25703 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:15.735958 25703 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 25, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:15.736752 25507 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:15.737504 25364 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:15.738126 25565 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:15.738332 25509 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:15.739073 25258 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 1 to 2, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 2 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:15.740018 25680 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:15.740559 25565 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:15.741959 25509 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:15.744122 25365 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:15.748878 25257 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:15.749070 25436 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 2.26, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:15.751134 25365 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:15.751681 25565 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:15.753897 25649 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.27: config changed from index -1 to 27, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:15.756695 25365 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.27: config changed from index -1 to 27, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:15.759883 25509 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:15.761022 25610 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
W20251210 21:05:15.762521 25403 consensus_peers.cc:597] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 681d997613474450ba6b9ee3de0f3f24. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:15.762617 25509 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.27: config changed from index -1 to 27, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
W20251210 21:05:15.764143 25406 consensus_peers.cc:597] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf7537ffe9f943b3a97e77adab5a1e23. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:15.764361 25258 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index -1 to 27, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251210 21:05:15.772732 25311 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251210 21:05:15.776520 25311 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 544270c4ded24ee2ae32f6aa4ec505c2, TS 4af27de48bec4590bfcd8d71056814eb -> TS 90c04d3cdd4b4cb18b2a265365b1b094 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:15.776605 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 544270c4ded24ee2ae32f6aa4ec505c2, TS 4af27de48bec4590bfcd8d71056814eb -> TS 90c04d3cdd4b4cb18b2a265365b1b094 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:15.779631 25311 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 4f1698d0d3bd4765bc0897771cb0ceee, TS 4af27de48bec4590bfcd8d71056814eb -> TS a6abef32d32c46478bc3f2b9aa589efb move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:15.779706 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 4f1698d0d3bd4765bc0897771cb0ceee, TS 4af27de48bec4590bfcd8d71056814eb -> TS a6abef32d32c46478bc3f2b9aa589efb move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:15.782693 25311 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet b65abf7cc0484fcda919a741e23928a7, TS 128e4ab89fe449e8b5fdc68590b7ac89 -> TS dbc0c6bd96c441e28d5a340d85fbe323 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:15.782768 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet b65abf7cc0484fcda919a741e23928a7, TS 128e4ab89fe449e8b5fdc68590b7ac89 -> TS dbc0c6bd96c441e28d5a340d85fbe323 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:15.785979 25311 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet e7e5750d29e94bad943f054f32b40746, TS 56e19ce44c0c4f08b1eb440e1ef3e9cd -> TS 90c04d3cdd4b4cb18b2a265365b1b094 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
W20251210 21:05:15.786072 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet e7e5750d29e94bad943f054f32b40746, TS 56e19ce44c0c4f08b1eb440e1ef3e9cd -> TS 90c04d3cdd4b4cb18b2a265365b1b094 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:16.178911 25507 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 2 index: 26. (index mismatch)
I20251210 21:05:16.179531 25616 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251210 21:05:16.183815 25435 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 2 index: 26. (index mismatch)
I20251210 21:05:16.184259 25938 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251210 21:05:16.206205 25942 ts_tablet_manager.cc:933] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: Initiating tablet copy from peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:16.206899 25942 tablet_copy_client.cc:323] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:39981
I20251210 21:05:16.222680 25450 tablet_copy_service.cc:140] P 4af27de48bec4590bfcd8d71056814eb: Received BeginTabletCopySession request for tablet bf7537ffe9f943b3a97e77adab5a1e23 from peer dbc0c6bd96c441e28d5a340d85fbe323 ({username='slave'} at 127.0.0.1:33316)
I20251210 21:05:16.222829 25450 tablet_copy_service.cc:161] P 4af27de48bec4590bfcd8d71056814eb: Beginning new tablet copy session on tablet bf7537ffe9f943b3a97e77adab5a1e23 from peer dbc0c6bd96c441e28d5a340d85fbe323 at {username='slave'} at 127.0.0.1:33316: session id = dbc0c6bd96c441e28d5a340d85fbe323-bf7537ffe9f943b3a97e77adab5a1e23
I20251210 21:05:16.225020 25450 tablet_copy_source_session.cc:215] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:16.226126 25942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7537ffe9f943b3a97e77adab5a1e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:16.227779 25945 ts_tablet_manager.cc:933] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: Initiating tablet copy from peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:16.228678 25945 tablet_copy_client.cc:323] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:39981
I20251210 21:05:16.232061 25942 tablet_copy_client.cc:806] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:16.232275 25942 tablet_copy_client.cc:670] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:16.234081 25943 tablet_copy_client.cc:1025] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:16.237430 25450 tablet_copy_service.cc:140] P 4af27de48bec4590bfcd8d71056814eb: Received BeginTabletCopySession request for tablet 681d997613474450ba6b9ee3de0f3f24 from peer a6abef32d32c46478bc3f2b9aa589efb ({username='slave'} at 127.0.0.1:33322)
I20251210 21:05:16.237545 25450 tablet_copy_service.cc:161] P 4af27de48bec4590bfcd8d71056814eb: Beginning new tablet copy session on tablet 681d997613474450ba6b9ee3de0f3f24 from peer a6abef32d32c46478bc3f2b9aa589efb at {username='slave'} at 127.0.0.1:33322: session id = a6abef32d32c46478bc3f2b9aa589efb-681d997613474450ba6b9ee3de0f3f24
I20251210 21:05:16.239581 25450 tablet_copy_source_session.cc:215] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:16.240604 25945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681d997613474450ba6b9ee3de0f3f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:16.245721 25945 tablet_copy_client.cc:806] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:16.245954 25945 tablet_copy_client.cc:670] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:16.247279 25946 tablet_copy_client.cc:1025] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:16.406641 25775 heartbeater.cc:499] Master 127.21.127.126:44841 was elected leader, sending a full tablet report...
I20251210 21:05:16.453233 25850 heartbeater.cc:499] Master 127.21.127.126:44841 was elected leader, sending a full tablet report...
I20251210 21:05:16.523520 25924 heartbeater.cc:499] Master 127.21.127.126:44841 was elected leader, sending a full tablet report...
I20251210 21:05:17.704941 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:17.707597 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:17.707789 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:18.234488 25942 tablet_copy_client.cc:538] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:18.238384 25942 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap starting.
I20251210 21:05:18.247653 25945 tablet_copy_client.cc:538] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:18.250130 25945 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap starting.
I20251210 21:05:18.277354 25942 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:18.277884 25942 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap complete.
I20251210 21:05:18.278165 25942 ts_tablet_manager.cc:1403] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent bootstrapping tablet: real 0.040s user 0.040s sys 0.000s
I20251210 21:05:18.278918 25942 raft_consensus.cc:359] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:18.279165 25942 raft_consensus.cc:740] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Initialized, Role: LEARNER
I20251210 21:05:18.279387 25942 consensus_queue.cc:260] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:18.280179 25942 ts_tablet_manager.cc:1434] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:18.280745 25450 tablet_copy_service.cc:342] P 4af27de48bec4590bfcd8d71056814eb: Request end of tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-bf7537ffe9f943b3a97e77adab5a1e23 received from {username='slave'} at 127.0.0.1:33316
I20251210 21:05:18.280861 25450 tablet_copy_service.cc:434] P 4af27de48bec4590bfcd8d71056814eb: ending tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-bf7537ffe9f943b3a97e77adab5a1e23 on tablet bf7537ffe9f943b3a97e77adab5a1e23 with peer dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:18.298133 25945 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=102 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:18.298558 25945 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap complete.
I20251210 21:05:18.299238 25945 ts_tablet_manager.cc:1403] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: Time spent bootstrapping tablet: real 0.049s user 0.032s sys 0.018s
I20251210 21:05:18.300002 25945 raft_consensus.cc:359] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:18.300189 25945 raft_consensus.cc:740] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Initialized, Role: LEARNER
I20251210 21:05:18.300354 25945 consensus_queue.cc:260] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:18.301051 25945 ts_tablet_manager.cc:1434] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:18.301628 25450 tablet_copy_service.cc:342] P 4af27de48bec4590bfcd8d71056814eb: Request end of tablet copy session a6abef32d32c46478bc3f2b9aa589efb-681d997613474450ba6b9ee3de0f3f24 received from {username='slave'} at 127.0.0.1:33322
I20251210 21:05:18.301745 25450 tablet_copy_service.cc:434] P 4af27de48bec4590bfcd8d71056814eb: ending tablet copy session a6abef32d32c46478bc3f2b9aa589efb-681d997613474450ba6b9ee3de0f3f24 on tablet 681d997613474450ba6b9ee3de0f3f24 with peer a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:18.306197 25750 raft_consensus.cc:1217] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.27->[2.28-2.28] Dedup: 2.28->[]
I20251210 21:05:18.733206 25825 raft_consensus.cc:1217] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.26->[2.27-2.27] Dedup: 2.27->[]
I20251210 21:05:18.783094 25956 raft_consensus.cc:1064] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
I20251210 21:05:18.784091 25956 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:18.786375 25507 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:18.786829 25750 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:18.786854 25365 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:18.787254 25950 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:18.787669 25957 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:18.787925 25956 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:18.791397 25950 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:18.792505 25507 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:18.795837 25365 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:18.796137 25750 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:18.799410 25256 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:18.806953 25435 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:18.808655 25750 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251210 21:05:18.808667 25365 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251210 21:05:18.809218 25957 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:18.809482 25950 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:18.811486 25957 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:18.812145 25365 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:18.812526 25750 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:18.815856 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet bf7537ffe9f943b3a97e77adab5a1e23 with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:18.815680 25258 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 29 to 30, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:18.819221 25499 tablet_service.cc:1552] Processing DeleteTablet for tablet bf7537ffe9f943b3a97e77adab5a1e23 with delete_type TABLET_DATA_TOMBSTONED (TS 56e19ce44c0c4f08b1eb440e1ef3e9cd not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:18.819943 25966 tablet_replica.cc:333] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:18.820185 25966 raft_consensus.cc:2243] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:18.820458 25966 raft_consensus.cc:2272] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:18.821633 25966 ts_tablet_manager.cc:1916] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:18.827209 25966 ts_tablet_manager.cc:1929] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.29
I20251210 21:05:18.827368 25966 log.cc:1199] T bf7537ffe9f943b3a97e77adab5a1e23 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/bf7537ffe9f943b3a97e77adab5a1e23
I20251210 21:05:18.828094 25247 catalog_manager.cc:4985] TS 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213): tablet bf7537ffe9f943b3a97e77adab5a1e23 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:19.315236 25956 raft_consensus.cc:1064] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: attempting to promote NON_VOTER a6abef32d32c46478bc3f2b9aa589efb to VOTER
I20251210 21:05:19.316061 25956 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:19.318285 25507 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:19.318285 25365 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:19.318379 25825 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:19.318930 25957 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:19.319197 25950 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:19.319651 25957 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:19.333096 25957 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index 27 to 28, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.333508 25509 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index 27 to 28, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.337801 25365 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index 27 to 28, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.338449 25257 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index 27 to 28, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.339075 25825 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index 27 to 28, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.346019 25436 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:19.348073 25509 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:19.348256 25825 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:19.348694 25956 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:19.349309 25968 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:19.351660 25950 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.29: config changed from index 28 to 29, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.352393 25509 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.354399 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 681d997613474450ba6b9ee3de0f3f24 with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:19.354635 25825 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:19.356230 25258 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 28 to 29, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:19.359591 25352 tablet_service.cc:1552] Processing DeleteTablet for tablet 681d997613474450ba6b9ee3de0f3f24 with delete_type TABLET_DATA_TOMBSTONED (TS 128e4ab89fe449e8b5fdc68590b7ac89 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:19.365993 25976 tablet_replica.cc:333] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:19.366210 25976 raft_consensus.cc:2243] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:19.366499 25976 raft_consensus.cc:2272] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:19.367295 25976 ts_tablet_manager.cc:1916] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:19.371620 25976 ts_tablet_manager.cc:1929] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.28
I20251210 21:05:19.371714 25976 log.cc:1199] T 681d997613474450ba6b9ee3de0f3f24 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/681d997613474450ba6b9ee3de0f3f24
I20251210 21:05:19.372321 25249 catalog_manager.cc:4985] TS 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): tablet 681d997613474450ba6b9ee3de0f3f24 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:19.708235 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:19.719722 25365 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:19.719969 25365 raft_consensus.cc:606] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Received request to transfer leadership to TS 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:19.725975 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:19.726194 25435 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:19.727350 25365 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:19.727547 25365 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:19.728580 25509 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:19.728736 25509 raft_consensus.cc:606] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:19.729120 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20251210 21:05:19.729305 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:19.740859 25977 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:19.740999 25977 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:19.741642 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43858
I20251210 21:05:19.741834 25435 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:19.741927 25435 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:19.743854 25435 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.744401 25435 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 3 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:19.744890 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 26 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:19.744890 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 26 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:19.745119 25365 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:19.745119 25509 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:19.745211 25365 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:19.745466 25365 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 2.26, Last appended by leader: 26, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.745775 25365 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:19.747987 25509 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 3.
I20251210 21:05:19.748238 25365 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 3.
I20251210 21:05:19.748485 25403 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:19.749015 25968 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:19.749151 25968 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:19.749372 25968 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 2.26, Last appended by leader: 26, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.752167 25257 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 2 to 3, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 3 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:19.871933 25955 raft_consensus.cc:993] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:19.872125 25955 raft_consensus.cc:1081] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Signalling peer 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:19.872830 25509 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:47102
I20251210 21:05:19.873000 25509 raft_consensus.cc:493] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:19.873106 25509 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:19.875034 25509 raft_consensus.cc:515] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.875839 25509 leader_election.cc:290] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:19.876166 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:19.876228 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:19.876336 25365 raft_consensus.cc:3055] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:19.876336 25435 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:19.876431 25365 raft_consensus.cc:740] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:19.876648 25365 consensus_queue.cc:260] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.876953 25365 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:19.879017 25435 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:19.879326 25365 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:19.879482 25477 leader_election.cc:304] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:19.879737 25602 raft_consensus.cc:2804] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:19.880065 25602 raft_consensus.cc:697] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:19.880290 25602 consensus_queue.cc:237] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.882474 25257 catalog_manager.cc:5654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 2 to 3, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 3 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:19.892942 25957 raft_consensus.cc:993] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:19.893074 25957 raft_consensus.cc:1081] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:19.893576 25365 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:19.893716 25365 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:19.893797 25365 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:19.895620 25365 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.896097 25365 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:19.896483 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:19.896629 25435 raft_consensus.cc:3055] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:19.896584 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:19.896696 25435 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:19.896804 25509 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:19.896915 25435 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.897213 25435 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:19.898749 25509 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:19.898921 25435 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:19.899201 25328 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:19.899470 25955 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:19.899601 25955 raft_consensus.cc:697] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:19.899883 25955 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:19.902292 25257 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 1 to 2, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 2 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:20.046296 25982 raft_consensus.cc:993] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:20.046444 25982 raft_consensus.cc:1081] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:20.046947 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43842
I20251210 21:05:20.047132 25435 raft_consensus.cc:493] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:20.047215 25435 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:20.049036 25435 raft_consensus.cc:515] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:20.049571 25435 leader_election.cc:290] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Requested vote from peers 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213), 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:20.050007 25365 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:20.050007 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:20.050334 25365 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:20.050464 25509 raft_consensus.cc:3055] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:20.050529 25509 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:20.050751 25509 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:20.051082 25509 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:20.053177 25365 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:20.053519 25509 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:20.053547 25405 leader_election.cc:304] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:20.053824 25957 raft_consensus.cc:2804] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:20.053956 25957 raft_consensus.cc:697] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:20.054193 25957 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:20.056571 25257 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 1 to 2, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:20.290288 25509 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:20.290833 25955 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.294762 25435 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:20.295979 25977 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.301934 25364 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 3 index: 27. (index mismatch)
I20251210 21:05:20.301947 25365 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251210 21:05:20.302402 25982 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:20.302740 25957 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.312121 25509 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 3 index: 27. (index mismatch)
I20251210 21:05:20.312637 25968 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.312885 25435 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251210 21:05:20.313339 25602 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:20.361747 25365 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:20.364306 25435 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251210 21:05:20.364355 25509 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251210 21:05:20.365092 25998 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.365437 25977 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.368542 25955 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.369144 25509 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.369411 25435 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.374205 25257 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index -1 to 27, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New cstate: current_term: 1 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
W20251210 21:05:20.374425 25331 consensus_peers.cc:597] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 -> Peer 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549): Couldn't send request to peer 90c04d3cdd4b4cb18b2a265365b1b094. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6e20042b566343a7acdbe06f4dcb0c86. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:20.377926 25435 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:20.380081 25365 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 1 index: 26. (index mismatch)
I20251210 21:05:20.380198 25509 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 1 index: 26. (index mismatch)
W20251210 21:05:20.380429 25403 consensus_peers.cc:597] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7e5750d29e94bad943f054f32b40746. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:20.380548 25968 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251210 21:05:20.380749 25993 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251210 21:05:20.383136 25993 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Committing config change with OpId 1.26: config changed from index -1 to 26, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:20.383955 25365 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Committing config change with OpId 1.26: config changed from index -1 to 26, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:20.387578 25258 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index -1 to 26, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New cstate: current_term: 1 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:20.388315 25365 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:20.390704 25435 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
W20251210 21:05:20.390770 25331 consensus_peers.cc:597] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 -> Peer 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549): Couldn't send request to peer 90c04d3cdd4b4cb18b2a265365b1b094. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 05b90891884a4db3bc8723b604cec5ea. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:20.391103 25507 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:20.391144 25955 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:20.391551 25998 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:20.388993 25509 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Committing config change with OpId 1.26: config changed from index -1 to 26, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:20.397379 25998 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.398005 25435 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.402200 25258 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index -1 to 28, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New cstate: current_term: 2 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:20.409472 25507 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.415269 25507 consensus_queue.cc:237] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:20.417718 25435 raft_consensus.cc:1275] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251210 21:05:20.418284 25365 raft_consensus.cc:1275] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251210 21:05:20.418583 25602 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.418864 25982 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.422017 25602 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:20.422572 25435 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:20.422727 25365 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:20.427817 25259 catalog_manager.cc:5654] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:20.427980 25435 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
W20251210 21:05:20.429888 25480 consensus_peers.cc:597] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 063c8d31eded49ca9a780697c68d485d. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:20.431156 25365 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251210 21:05:20.431504 25750 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251210 21:05:20.431736 25957 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:20.433883 25957 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:20.436487 25968 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 30 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
W20251210 21:05:20.440460 25406 consensus_peers.cc:597] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb -> Peer 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549): Couldn't send request to peer 90c04d3cdd4b4cb18b2a265365b1b094. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf7537ffe9f943b3a97e77adab5a1e23. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:20.440531 25750 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.441577 25257 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 30 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:20.444273 25364 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:20.446905 25365 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:20.449466 25435 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251210 21:05:20.449935 25998 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.450433 25507 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251210 21:05:20.450915 25998 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.453009 25955 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:20.453697 25435 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
W20251210 21:05:20.456161 25331 consensus_peers.cc:597] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f1698d0d3bd4765bc0897771cb0ceee. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:20.456619 25507 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:20.458618 25258 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index -1 to 27, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:20.516292 25507 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:20.516750 25968 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.520481 25365 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:20.521082 25957 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:20.787047 26027 ts_tablet_manager.cc:933] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: Initiating tablet copy from peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:20.788158 26027 tablet_copy_client.cc:323] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:41411
I20251210 21:05:20.796136 25375 tablet_copy_service.cc:140] P 128e4ab89fe449e8b5fdc68590b7ac89: Received BeginTabletCopySession request for tablet 6e20042b566343a7acdbe06f4dcb0c86 from peer 90c04d3cdd4b4cb18b2a265365b1b094 ({username='slave'} at 127.0.0.1:55596)
I20251210 21:05:20.796310 25375 tablet_copy_service.cc:161] P 128e4ab89fe449e8b5fdc68590b7ac89: Beginning new tablet copy session on tablet 6e20042b566343a7acdbe06f4dcb0c86 from peer 90c04d3cdd4b4cb18b2a265365b1b094 at {username='slave'} at 127.0.0.1:55596: session id = 90c04d3cdd4b4cb18b2a265365b1b094-6e20042b566343a7acdbe06f4dcb0c86
I20251210 21:05:20.798662 25375 tablet_copy_source_session.cc:215] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:20.799839 26027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20042b566343a7acdbe06f4dcb0c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:20.805122 26027 tablet_copy_client.cc:806] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:20.805390 26027 tablet_copy_client.cc:670] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:20.807108 26028 tablet_copy_client.cc:1025] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:20.810077 26031 ts_tablet_manager.cc:933] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: Initiating tablet copy from peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:20.811089 26031 tablet_copy_client.cc:323] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:39981
I20251210 21:05:20.811710 25450 tablet_copy_service.cc:140] P 4af27de48bec4590bfcd8d71056814eb: Received BeginTabletCopySession request for tablet e7e5750d29e94bad943f054f32b40746 from peer a6abef32d32c46478bc3f2b9aa589efb ({username='slave'} at 127.0.0.1:33322)
I20251210 21:05:20.811836 25450 tablet_copy_service.cc:161] P 4af27de48bec4590bfcd8d71056814eb: Beginning new tablet copy session on tablet e7e5750d29e94bad943f054f32b40746 from peer a6abef32d32c46478bc3f2b9aa589efb at {username='slave'} at 127.0.0.1:33322: session id = a6abef32d32c46478bc3f2b9aa589efb-e7e5750d29e94bad943f054f32b40746
I20251210 21:05:20.814128 25450 tablet_copy_source_session.cc:215] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:20.815625 26031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e5750d29e94bad943f054f32b40746. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:20.821070 26031 tablet_copy_client.cc:806] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:20.821281 26031 tablet_copy_client.cc:670] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:20.822892 26032 tablet_copy_client.cc:1025] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:20.890404 26033 ts_tablet_manager.cc:933] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: Initiating tablet copy from peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:20.891184 26033 tablet_copy_client.cc:323] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:41411
I20251210 21:05:20.891875 25375 tablet_copy_service.cc:140] P 128e4ab89fe449e8b5fdc68590b7ac89: Received BeginTabletCopySession request for tablet 05b90891884a4db3bc8723b604cec5ea from peer 90c04d3cdd4b4cb18b2a265365b1b094 ({username='slave'} at 127.0.0.1:55596)
I20251210 21:05:20.892033 25375 tablet_copy_service.cc:161] P 128e4ab89fe449e8b5fdc68590b7ac89: Beginning new tablet copy session on tablet 05b90891884a4db3bc8723b604cec5ea from peer 90c04d3cdd4b4cb18b2a265365b1b094 at {username='slave'} at 127.0.0.1:55596: session id = 90c04d3cdd4b4cb18b2a265365b1b094-05b90891884a4db3bc8723b604cec5ea
I20251210 21:05:20.894346 25375 tablet_copy_source_session.cc:215] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:20.895481 26033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b90891884a4db3bc8723b604cec5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:20.901433 26033 tablet_copy_client.cc:806] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:20.901731 26033 tablet_copy_client.cc:670] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:20.903205 26034 tablet_copy_client.cc:1025] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:20.974960 26035 ts_tablet_manager.cc:933] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Initiating tablet copy from peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:20.975656 26035 tablet_copy_client.cc:323] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:41411
I20251210 21:05:20.982698 25375 tablet_copy_service.cc:140] P 128e4ab89fe449e8b5fdc68590b7ac89: Received BeginTabletCopySession request for tablet 4f1698d0d3bd4765bc0897771cb0ceee from peer dbc0c6bd96c441e28d5a340d85fbe323 ({username='slave'} at 127.0.0.1:55598)
I20251210 21:05:20.982831 25375 tablet_copy_service.cc:161] P 128e4ab89fe449e8b5fdc68590b7ac89: Beginning new tablet copy session on tablet 4f1698d0d3bd4765bc0897771cb0ceee from peer dbc0c6bd96c441e28d5a340d85fbe323 at {username='slave'} at 127.0.0.1:55598: session id = dbc0c6bd96c441e28d5a340d85fbe323-4f1698d0d3bd4765bc0897771cb0ceee
I20251210 21:05:20.985271 25375 tablet_copy_source_session.cc:215] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:20.986364 26035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f1698d0d3bd4765bc0897771cb0ceee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:20.991818 26035 tablet_copy_client.cc:806] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:20.992079 26035 tablet_copy_client.cc:670] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:20.993625 26036 tablet_copy_client.cc:1025] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:21.019203 26038 ts_tablet_manager.cc:933] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: Initiating tablet copy from peer 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:21.019899 26038 tablet_copy_client.cc:323] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:36213
I20251210 21:05:21.025460 25524 tablet_copy_service.cc:140] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Received BeginTabletCopySession request for tablet 063c8d31eded49ca9a780697c68d485d from peer dbc0c6bd96c441e28d5a340d85fbe323 ({username='slave'} at 127.0.0.1:55876)
I20251210 21:05:21.025636 25524 tablet_copy_service.cc:161] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Beginning new tablet copy session on tablet 063c8d31eded49ca9a780697c68d485d from peer dbc0c6bd96c441e28d5a340d85fbe323 at {username='slave'} at 127.0.0.1:55876: session id = dbc0c6bd96c441e28d5a340d85fbe323-063c8d31eded49ca9a780697c68d485d
I20251210 21:05:21.027626 25524 tablet_copy_source_session.cc:215] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:21.028703 26038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063c8d31eded49ca9a780697c68d485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:21.034231 26038 tablet_copy_client.cc:806] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:21.034504 26038 tablet_copy_client.cc:670] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:21.036248 26039 tablet_copy_client.cc:1025] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:21.053954 26041 ts_tablet_manager.cc:933] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: Initiating tablet copy from peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:21.054656 26041 tablet_copy_client.cc:323] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:39981
I20251210 21:05:21.061641 25450 tablet_copy_service.cc:140] P 4af27de48bec4590bfcd8d71056814eb: Received BeginTabletCopySession request for tablet bf7537ffe9f943b3a97e77adab5a1e23 from peer 90c04d3cdd4b4cb18b2a265365b1b094 ({username='slave'} at 127.0.0.1:33330)
I20251210 21:05:21.061777 25450 tablet_copy_service.cc:161] P 4af27de48bec4590bfcd8d71056814eb: Beginning new tablet copy session on tablet bf7537ffe9f943b3a97e77adab5a1e23 from peer 90c04d3cdd4b4cb18b2a265365b1b094 at {username='slave'} at 127.0.0.1:33330: session id = 90c04d3cdd4b4cb18b2a265365b1b094-bf7537ffe9f943b3a97e77adab5a1e23
I20251210 21:05:21.064100 25450 tablet_copy_source_session.cc:215] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:21.065091 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7537ffe9f943b3a97e77adab5a1e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:21.069418 26041 tablet_copy_client.cc:806] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:21.069705 26041 tablet_copy_client.cc:670] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:21.073167 26042 tablet_copy_client.cc:1025] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:21.729628 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:21.733124 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:21.733357 25364 raft_consensus.cc:606] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Received request to transfer leadership to TS 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:21.734369 25436 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:21.734526 25436 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:21.735615 25436 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:21.735792 25436 raft_consensus.cc:606] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Received request to transfer leadership to TS a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:21.736828 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:21.736984 25507 raft_consensus.cc:606] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:21.737367 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20251210 21:05:21.737588 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:22.015158 26045 raft_consensus.cc:993] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:22.015340 26045 raft_consensus.cc:1081] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:22.015937 25436 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43842
I20251210 21:05:22.016108 25436 raft_consensus.cc:493] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:22.016211 25436 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:22.018659 25436 raft_consensus.cc:515] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:22.019285 25436 leader_election.cc:290] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:22.019624 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:22.019755 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:22.019915 25507 raft_consensus.cc:3055] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:22.019982 25507 raft_consensus.cc:740] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:22.020243 25507 consensus_queue.cc:260] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:22.020515 25507 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:22.022627 25507 raft_consensus.cc:2468] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:22.019806 25364 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:22.023030 25403 leader_election.cc:304] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:22.023319 25957 raft_consensus.cc:2804] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:22.023463 25957 raft_consensus.cc:697] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:22.023710 25957 consensus_queue.cc:237] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:22.025133 25364 raft_consensus.cc:2468] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 2.
I20251210 21:05:22.027074 25259 catalog_manager.cc:5654] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 1 to 2, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:22.046306 25993 raft_consensus.cc:993] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower a6abef32d32c46478bc3f2b9aa589efb to start an election
I20251210 21:05:22.046480 25993 raft_consensus.cc:1081] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Signalling peer a6abef32d32c46478bc3f2b9aa589efb to start an election
I20251210 21:05:22.047116 25825 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24"
dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
from {username='slave'} at 127.0.0.1:46918
I20251210 21:05:22.047329 25825 raft_consensus.cc:493] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:22.047430 25825 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:22.049959 25825 raft_consensus.cc:515] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:22.050727 25825 leader_election.cc:290] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [CANDIDATE]: Term 3 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:22.051076 25436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "a6abef32d32c46478bc3f2b9aa589efb" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:22.051234 25436 raft_consensus.cc:3055] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:22.051296 25436 raft_consensus.cc:740] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:22.051556 25436 consensus_queue.cc:260] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:22.051905 25436 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:22.054769 25436 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6abef32d32c46478bc3f2b9aa589efb in term 3.
I20251210 21:05:22.055178 25788 leader_election.cc:304] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, a6abef32d32c46478bc3f2b9aa589efb; no voters:
I20251210 21:05:22.055819 26053 raft_consensus.cc:2804] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:22.056408 26053 raft_consensus.cc:697] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEADER]: Becoming Leader. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Running, Role: LEADER
I20251210 21:05:22.056739 26053 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:22.059504 25259 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb reported cstate change: term changed from 2 to 3, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69). New cstate: current_term: 3 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:22.061059 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "681d997613474450ba6b9ee3de0f3f24" candidate_uuid: "a6abef32d32c46478bc3f2b9aa589efb" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:22.061233 25507 raft_consensus.cc:3060] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:22.063649 25507 raft_consensus.cc:2468] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6abef32d32c46478bc3f2b9aa589efb in term 3.
I20251210 21:05:22.090306 25998 raft_consensus.cc:993] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:22.090512 25998 raft_consensus.cc:1081] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Signalling peer 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:22.091140 25507 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:47102
I20251210 21:05:22.091297 25507 raft_consensus.cc:493] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:22.091382 25507 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:22.093546 25507 raft_consensus.cc:515] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:22.094139 25507 leader_election.cc:290] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 2 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:22.094666 25436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:22.094847 25436 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:22.097081 25436 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 2.
I20251210 21:05:22.097326 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:22.097483 25364 raft_consensus.cc:3055] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:22.097438 25477 leader_election.cc:304] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:22.097565 25364 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:22.097729 26045 raft_consensus.cc:2804] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:22.097847 26045 raft_consensus.cc:697] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:22.097795 25364 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:22.098147 25364 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:22.098096 26045 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:22.100242 25364 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 2.
I20251210 21:05:22.100917 25259 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 1 to 2, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 2 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:22.108933 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:22.109113 25507 raft_consensus.cc:606] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 LEADER]: Received request to transfer leadership
I20251210 21:05:22.116865 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:22.117048 25507 raft_consensus.cc:606] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 LEADER]: Received request to transfer leadership
W20251210 21:05:22.117403 25247 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 4f1698d0d3bd4765bc0897771cb0ceee already in progress [suppressed 3 similar messages]
I20251210 21:05:22.118538 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:22.118672 25507 raft_consensus.cc:606] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 LEADER]: Received request to transfer leadership
W20251210 21:05:22.119113 25311 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 4f1698d0d3bd4765bc0897771cb0ceee already in progress
W20251210 21:05:22.119180 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 4f1698d0d3bd4765bc0897771cb0ceee already in progress
I20251210 21:05:22.198844 25957 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:22.199064 25993 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:22.199726 25365 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:22.199960 25365 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:22.200062 25365 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:22.202530 25365 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:22.203102 25365 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 4 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:22.203302 25436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 4 candidate_status { last_received { term: 3 index: 27 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:22.203507 25436 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Stepping down as leader of term 3
I20251210 21:05:22.203596 25436 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:22.203603 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 4 candidate_status { last_received { term: 3 index: 27 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:22.203792 25507 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:22.203907 25436 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 3.27, Last appended by leader: 27, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:22.204267 25436 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:22.206410 25507 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 4.
I20251210 21:05:22.206737 25328 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:22.206984 25436 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 4.
I20251210 21:05:22.207085 25998 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Leader election won for term 4
I20251210 21:05:22.207255 25998 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:22.207499 25998 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 3.27, Last appended by leader: 27, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:22.210013 25259 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 3 to 4, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 4 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:22.443979 25507 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:22.444434 26053 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:22.448927 25436 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:22.449375 26053 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:22.519814 25365 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:22.520364 26045 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:22.522682 26044 raft_consensus.cc:993] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:22.522934 26045 raft_consensus.cc:1081] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:22.524111 25363 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39018
I20251210 21:05:22.524276 25436 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:22.525220 26045 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20251210 21:05:22.525441 25480 consensus_peers.cc:597] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:22.535164 25363 raft_consensus.cc:493] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:22.535305 25363 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:22.537485 25363 raft_consensus.cc:515] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:22.538462 25363 leader_election.cc:290] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 3 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:22.538841 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:22.539001 25509 raft_consensus.cc:3055] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:22.538942 25436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:22.539063 25509 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:22.539127 25436 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:22.539283 25509 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:22.539623 25509 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:22.541654 25436 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 3.
I20251210 21:05:22.541899 25509 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 3.
I20251210 21:05:22.542258 25328 leader_election.cc:304] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:22.542505 25998 raft_consensus.cc:2804] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:22.542626 25998 raft_consensus.cc:697] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:22.542845 25998 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:22.545291 25259 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 2 to 3, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 3 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:22.548151 25509 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:22.548631 25993 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:22.552057 25364 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:22.552495 25957 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:22.660382 25509 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 27. Preceding OpId from leader: term: 4 index: 28. (index mismatch)
I20251210 21:05:22.660871 25998 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:22.664147 25436 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 27. Preceding OpId from leader: term: 4 index: 28. (index mismatch)
I20251210 21:05:22.664652 25977 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:22.807479 26027 tablet_copy_client.cc:538] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:22.810608 26027 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap starting.
I20251210 21:05:22.823266 26031 tablet_copy_client.cc:538] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:22.826576 26031 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap starting.
I20251210 21:05:22.851511 26027 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:22.852016 26027 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap complete.
I20251210 21:05:22.852380 26027 ts_tablet_manager.cc:1403] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent bootstrapping tablet: real 0.042s user 0.043s sys 0.001s
I20251210 21:05:22.853221 26027 raft_consensus.cc:359] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:22.853567 26027 raft_consensus.cc:740] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 90c04d3cdd4b4cb18b2a265365b1b094, State: Initialized, Role: LEARNER
I20251210 21:05:22.853766 26027 consensus_queue.cc:260] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:22.859995 26027 ts_tablet_manager.cc:1434] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20251210 21:05:22.860762 25375 tablet_copy_service.cc:342] P 128e4ab89fe449e8b5fdc68590b7ac89: Request end of tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-6e20042b566343a7acdbe06f4dcb0c86 received from {username='slave'} at 127.0.0.1:55596
I20251210 21:05:22.860894 25375 tablet_copy_service.cc:434] P 128e4ab89fe449e8b5fdc68590b7ac89: ending tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-6e20042b566343a7acdbe06f4dcb0c86 on tablet 6e20042b566343a7acdbe06f4dcb0c86 with peer 90c04d3cdd4b4cb18b2a265365b1b094
I20251210 21:05:22.866150 26031 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=0 applied=26 ignored=0} inserts{seen=101 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:22.866482 26031 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap complete.
I20251210 21:05:22.866765 26031 ts_tablet_manager.cc:1403] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: Time spent bootstrapping tablet: real 0.040s user 0.034s sys 0.006s
I20251210 21:05:22.867415 26031 raft_consensus.cc:359] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:22.867602 26031 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Initialized, Role: LEARNER
I20251210 21:05:22.867835 26031 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:22.868300 26031 ts_tablet_manager.cc:1434] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:22.868920 25450 tablet_copy_service.cc:342] P 4af27de48bec4590bfcd8d71056814eb: Request end of tablet copy session a6abef32d32c46478bc3f2b9aa589efb-e7e5750d29e94bad943f054f32b40746 received from {username='slave'} at 127.0.0.1:33322
I20251210 21:05:22.869029 25450 tablet_copy_service.cc:434] P 4af27de48bec4590bfcd8d71056814eb: ending tablet copy session a6abef32d32c46478bc3f2b9aa589efb-e7e5750d29e94bad943f054f32b40746 on tablet e7e5750d29e94bad943f054f32b40746 with peer a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:22.903566 26033 tablet_copy_client.cc:538] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:22.907421 26033 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap starting.
I20251210 21:05:22.928241 25509 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:22.928836 26071 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251210 21:05:22.934280 25331 consensus_peers.cc:597] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:22.934804 25435 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:22.936514 26071 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:22.945093 26033 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=99 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:22.945426 26033 tablet_bootstrap.cc:492] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap complete.
I20251210 21:05:22.945652 26033 ts_tablet_manager.cc:1403] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent bootstrapping tablet: real 0.038s user 0.033s sys 0.006s
I20251210 21:05:22.946285 26033 raft_consensus.cc:359] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:22.946461 26033 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 90c04d3cdd4b4cb18b2a265365b1b094, State: Initialized, Role: LEARNER
I20251210 21:05:22.946640 26033 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:22.947038 26033 ts_tablet_manager.cc:1434] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:05:22.948026 25375 tablet_copy_service.cc:342] P 128e4ab89fe449e8b5fdc68590b7ac89: Request end of tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-05b90891884a4db3bc8723b604cec5ea received from {username='slave'} at 127.0.0.1:55596
I20251210 21:05:22.948164 25375 tablet_copy_service.cc:434] P 128e4ab89fe449e8b5fdc68590b7ac89: ending tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-05b90891884a4db3bc8723b604cec5ea on tablet 05b90891884a4db3bc8723b604cec5ea with peer 90c04d3cdd4b4cb18b2a265365b1b094
I20251210 21:05:22.994007 26035 tablet_copy_client.cc:538] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:22.996037 25899 raft_consensus.cc:1217] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251210 21:05:22.997547 26035 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap starting.
I20251210 21:05:23.033145 26035 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=89 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:23.033555 26035 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap complete.
I20251210 21:05:23.033815 26035 ts_tablet_manager.cc:1403] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Time spent bootstrapping tablet: real 0.036s user 0.031s sys 0.005s
I20251210 21:05:23.034545 26035 raft_consensus.cc:359] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:23.034691 26035 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Initialized, Role: LEARNER
I20251210 21:05:23.034858 26035 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:23.035683 26035 ts_tablet_manager.cc:1434] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20251210 21:05:23.036295 25375 tablet_copy_service.cc:342] P 128e4ab89fe449e8b5fdc68590b7ac89: Request end of tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-4f1698d0d3bd4765bc0897771cb0ceee received from {username='slave'} at 127.0.0.1:55598
I20251210 21:05:23.036389 25375 tablet_copy_service.cc:434] P 128e4ab89fe449e8b5fdc68590b7ac89: ending tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-4f1698d0d3bd4765bc0897771cb0ceee on tablet 4f1698d0d3bd4765bc0897771cb0ceee with peer dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:23.036563 26038 tablet_copy_client.cc:538] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:23.039862 26038 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap starting.
I20251210 21:05:23.073532 26041 tablet_copy_client.cc:538] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:23.075963 26041 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap starting.
I20251210 21:05:23.079196 26038 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=109 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:23.079557 26038 tablet_bootstrap.cc:492] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap complete.
I20251210 21:05:23.079818 26038 ts_tablet_manager.cc:1403] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: Time spent bootstrapping tablet: real 0.040s user 0.027s sys 0.007s
I20251210 21:05:23.080536 26038 raft_consensus.cc:359] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:23.080739 26038 raft_consensus.cc:740] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Initialized, Role: LEARNER
I20251210 21:05:23.080932 26038 consensus_queue.cc:260] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:23.081413 26038 ts_tablet_manager.cc:1434] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:23.082090 25524 tablet_copy_service.cc:342] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Request end of tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-063c8d31eded49ca9a780697c68d485d received from {username='slave'} at 127.0.0.1:55876
I20251210 21:05:23.082206 25524 tablet_copy_service.cc:434] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: ending tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-063c8d31eded49ca9a780697c68d485d on tablet 063c8d31eded49ca9a780697c68d485d with peer dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:23.116060 26041 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:23.116442 26041 tablet_bootstrap.cc:492] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap complete.
I20251210 21:05:23.116722 26041 ts_tablet_manager.cc:1403] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent bootstrapping tablet: real 0.041s user 0.032s sys 0.007s
I20251210 21:05:23.117362 26041 raft_consensus.cc:359] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:23.117574 26041 raft_consensus.cc:740] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 90c04d3cdd4b4cb18b2a265365b1b094, State: Initialized, Role: LEARNER
I20251210 21:05:23.117786 26041 consensus_queue.cc:260] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:23.118285 26041 ts_tablet_manager.cc:1434] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:05:23.118909 25450 tablet_copy_service.cc:342] P 4af27de48bec4590bfcd8d71056814eb: Request end of tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-bf7537ffe9f943b3a97e77adab5a1e23 received from {username='slave'} at 127.0.0.1:33330
I20251210 21:05:23.119024 25450 tablet_copy_service.cc:434] P 4af27de48bec4590bfcd8d71056814eb: ending tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-bf7537ffe9f943b3a97e77adab5a1e23 on tablet bf7537ffe9f943b3a97e77adab5a1e23 with peer 90c04d3cdd4b4cb18b2a265365b1b094
I20251210 21:05:23.337586 25899 raft_consensus.cc:1217] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.27->[2.28-2.28] Dedup: 2.28->[]
I20251210 21:05:23.385705 25825 raft_consensus.cc:1217] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.25->[1.26-1.26] Dedup: 1.26->[]
I20251210 21:05:23.440698 25899 raft_consensus.cc:1217] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.30->[2.31-2.31] Dedup: 2.31->[]
I20251210 21:05:23.471292 26071 raft_consensus.cc:1064] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: attempting to promote NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 to VOTER
I20251210 21:05:23.472148 26071 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:23.474457 25899 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251210 21:05:23.474716 25435 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251210 21:05:23.474846 25509 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251210 21:05:23.474965 25998 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:23.475217 25977 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:23.475732 25998 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:23.488883 26083 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:23.489490 25899 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:23.490412 25509 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:23.493163 25435 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:23.494346 25257 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 27 to 28, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:23.503573 25363 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:23.505309 25509 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251210 21:05:23.505330 25899 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251210 21:05:23.505959 25977 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:23.506214 26083 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:23.508970 26071 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:23.509562 25507 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:23.512591 25249 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 6e20042b566343a7acdbe06f4dcb0c86 with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:23.514045 25899 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:23.514219 25259 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New cstate: current_term: 1 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:23.517997 25422 tablet_service.cc:1552] Processing DeleteTablet for tablet 6e20042b566343a7acdbe06f4dcb0c86 with delete_type TABLET_DATA_TOMBSTONED (TS 4af27de48bec4590bfcd8d71056814eb not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:23.519387 26090 tablet_replica.cc:333] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:23.519716 26090 raft_consensus.cc:2243] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:23.520009 26090 raft_consensus.cc:2272] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:23.521695 26090 ts_tablet_manager.cc:1916] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:23.527243 26090 ts_tablet_manager.cc:1929] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251210 21:05:23.527372 26090 log.cc:1199] T 6e20042b566343a7acdbe06f4dcb0c86 P 4af27de48bec4590bfcd8d71056814eb: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/6e20042b566343a7acdbe06f4dcb0c86
I20251210 21:05:23.528010 25247 catalog_manager.cc:4985] TS 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): tablet 6e20042b566343a7acdbe06f4dcb0c86 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:23.529443 25750 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 LEARNER]: Advancing to term 3
I20251210 21:05:23.531862 25750 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:23.532527 26083 consensus_queue.cc:1050] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:23.535545 26071 raft_consensus.cc:1064] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
I20251210 21:05:23.536309 26071 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 28, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:23.539549 25750 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:23.539779 25435 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:23.539544 25509 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:23.540243 26083 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:23.540349 25750 raft_consensus.cc:1217] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251210 21:05:23.540699 26083 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:23.541427 26083 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:23.542717 26083 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Committing config change with OpId 3.30: config changed from index 27 to 30, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:23.543339 25749 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 27 to 30, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:23.546774 25435 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 27 to 30, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:23.547655 25509 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 27 to 30, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:23.548079 25258 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 27 to 30, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:23.554593 25364 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:23.556677 25437 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:23.556677 25749 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:23.557215 26071 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:23.557503 25998 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:23.559659 25998 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:23.560169 25437 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:23.560612 25749 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:23.563592 25249 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 4f1698d0d3bd4765bc0897771cb0ceee with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:23.564653 25259 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New cstate: current_term: 3 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:23.568084 25499 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f1698d0d3bd4765bc0897771cb0ceee with delete_type TABLET_DATA_TOMBSTONED (TS 56e19ce44c0c4f08b1eb440e1ef3e9cd not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:23.578138 26093 tablet_replica.cc:333] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:23.578380 26093 raft_consensus.cc:2243] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:23.578732 26093 raft_consensus.cc:2272] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:23.579560 26093 ts_tablet_manager.cc:1916] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:23.585564 26093 ts_tablet_manager.cc:1929] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.30
I20251210 21:05:23.585719 26093 log.cc:1199] T 4f1698d0d3bd4765bc0897771cb0ceee P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/4f1698d0d3bd4765bc0897771cb0ceee
I20251210 21:05:23.586509 25247 catalog_manager.cc:4985] TS 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213): tablet 4f1698d0d3bd4765bc0897771cb0ceee (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:23.737927 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:23.741492 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:23.741735 25364 raft_consensus.cc:606] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Received request to transfer leadership to TS 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:23.742679 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:23.742862 25364 raft_consensus.cc:606] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Received request to transfer leadership to TS dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:23.743896 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:23.744059 25364 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Received request to transfer leadership to TS 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:23.744930 25437 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:23.745081 25437 raft_consensus.cc:606] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:23.745986 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:23.746140 25435 raft_consensus.cc:606] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:23.746462 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20251210 21:05:23.746691 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:23.773917 26083 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:23.774125 25977 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Signalling peer 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:23.774791 25509 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:47102
I20251210 21:05:23.774962 25509 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:23.775060 25509 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:23.777678 25509 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:23.778312 25509 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 5 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:23.778748 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 28 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:23.778929 25364 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Stepping down as leader of term 4
I20251210 21:05:23.778992 25364 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:23.779232 25364 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 4.28, Last appended by leader: 28, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:23.779551 25364 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:23.779595 25437 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 28 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:23.779755 25437 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:23.781615 25364 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 5.
I20251210 21:05:23.781713 25437 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 5.
I20251210 21:05:23.781996 25479 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:23.782275 26045 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 FOLLOWER]: Leader election won for term 5
I20251210 21:05:23.782629 26045 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:23.782919 26045 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 4.28, Last appended by leader: 28, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } }
I20251210 21:05:23.785620 25256 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 4 to 5, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 5 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:23.857146 26096 raft_consensus.cc:1064] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: attempting to promote NON_VOTER a6abef32d32c46478bc3f2b9aa589efb to VOTER
W20251210 21:05:23.857302 26096 raft_consensus.cc:1068] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Unable to promote non-voter a6abef32d32c46478bc3f2b9aa589efb: Service unavailable: leader transfer in progress
I20251210 21:05:23.904168 25977 raft_consensus.cc:1064] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: attempting to promote NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 to VOTER
W20251210 21:05:23.904310 25977 raft_consensus.cc:1068] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: Unable to promote non-voter 90c04d3cdd4b4cb18b2a265365b1b094: Service unavailable: leader transfer in progress
I20251210 21:05:24.003612 25993 raft_consensus.cc:1064] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: attempting to promote NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 to VOTER
I20251210 21:05:24.004463 25993 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:24.006820 25364 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251210 21:05:24.006901 25899 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251210 21:05:24.006845 25749 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251210 21:05:24.007339 25993 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:24.007755 26096 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:24.008203 26098 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:24.016984 26096 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.017691 25364 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.018764 25749 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.022559 25899 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.022923 25256 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.028362 25435 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:24.030721 25899 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251210 21:05:24.031628 26098 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:24.031845 25749 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251210 21:05:24.032373 26098 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:24.034412 26096 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.035522 25749 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.035293 25899 raft_consensus.cc:2955] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.038341 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet bf7537ffe9f943b3a97e77adab5a1e23 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:24.040613 25259 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.043905 25352 tablet_service.cc:1552] Processing DeleteTablet for tablet bf7537ffe9f943b3a97e77adab5a1e23 with delete_type TABLET_DATA_TOMBSTONED (TS 128e4ab89fe449e8b5fdc68590b7ac89 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:24.044606 26103 tablet_replica.cc:333] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:24.044814 26103 raft_consensus.cc:2243] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:24.045239 26103 raft_consensus.cc:2272] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:24.046109 26103 ts_tablet_manager.cc:1916] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:24.051350 26103 ts_tablet_manager.cc:1929] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.32
I20251210 21:05:24.051484 26103 log.cc:1199] T bf7537ffe9f943b3a97e77adab5a1e23 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/bf7537ffe9f943b3a97e77adab5a1e23
I20251210 21:05:24.052150 25249 catalog_manager.cc:4985] TS 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): tablet bf7537ffe9f943b3a97e77adab5a1e23 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:24.072153 26104 raft_consensus.cc:1064] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
I20251210 21:05:24.072906 26104 consensus_queue.cc:237] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:24.074757 25749 raft_consensus.cc:1275] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 LEARNER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251210 21:05:24.075117 25435 raft_consensus.cc:1275] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251210 21:05:24.075471 25364 raft_consensus.cc:1275] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251210 21:05:24.075578 26104 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:24.075825 26106 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:24.076050 26094 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:24.079186 26094 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.079712 25364 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.083671 25256 catalog_manager.cc:5654] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 27 to 28, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.087639 25435 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.090545 25977 raft_consensus.cc:993] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:24.090667 25977 raft_consensus.cc:1081] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Signalling peer 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:24.090526 25749 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.091187 25507 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:47102
I20251210 21:05:24.091351 25507 raft_consensus.cc:493] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:24.091470 25507 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.093624 25507 raft_consensus.cc:515] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:24.094625 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:24.094863 25364 raft_consensus.cc:3055] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:24.094980 25364 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:24.095305 25364 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:24.095703 25364 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.096267 25507 leader_election.cc:290] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:24.097751 25364 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:24.098070 25437 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:24.098217 25437 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.098274 25479 leader_election.cc:304] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:24.098524 26104 raft_consensus.cc:2804] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:24.098646 26104 raft_consensus.cc:697] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:24.098842 25507 consensus_queue.cc:237] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:24.098856 26104 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:24.100577 25437 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:24.101068 25364 raft_consensus.cc:1275] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251210 21:05:24.101771 25749 raft_consensus.cc:1275] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251210 21:05:24.102344 26106 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:24.102757 26094 consensus_queue.cc:1048] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:24.105006 26106 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.106532 25364 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.108888 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 063c8d31eded49ca9a780697c68d485d with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:24.108901 25749 raft_consensus.cc:2955] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.109565 25258 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 2 to 3, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 3 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:24.113195 25256 catalog_manager.cc:5654] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 28 to 29, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New cstate: current_term: 1 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:24.116526 25422 tablet_service.cc:1552] Processing DeleteTablet for tablet 063c8d31eded49ca9a780697c68d485d with delete_type TABLET_DATA_TOMBSTONED (TS 4af27de48bec4590bfcd8d71056814eb not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:24.116997 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:24.117143 25507 raft_consensus.cc:606] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Received request to transfer leadership
I20251210 21:05:24.117784 26111 tablet_replica.cc:333] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:24.117966 26111 raft_consensus.cc:2243] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:24.118242 26111 raft_consensus.cc:2272] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:24.118968 26111 ts_tablet_manager.cc:1916] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:24.120239 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:24.120373 25507 raft_consensus.cc:606] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Received request to transfer leadership
I20251210 21:05:24.121490 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:24.121611 25507 raft_consensus.cc:606] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Received request to transfer leadership
W20251210 21:05:24.121975 25311 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 05b90891884a4db3bc8723b604cec5ea already in progress
W20251210 21:05:24.122035 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 05b90891884a4db3bc8723b604cec5ea already in progress
I20251210 21:05:24.123678 26111 ts_tablet_manager.cc:1929] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251210 21:05:24.123768 26111 log.cc:1199] T 063c8d31eded49ca9a780697c68d485d P 4af27de48bec4590bfcd8d71056814eb: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/063c8d31eded49ca9a780697c68d485d
I20251210 21:05:24.124284 25247 catalog_manager.cc:4985] TS 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): tablet 063c8d31eded49ca9a780697c68d485d (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:24.131228 26096 raft_consensus.cc:993] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:24.131363 26096 raft_consensus.cc:1081] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:24.131970 25364 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:24.132143 25364 raft_consensus.cc:493] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:24.132241 25364 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.134209 25364 raft_consensus.cc:515] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:24.134842 25364 leader_election.cc:290] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 3 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:24.135279 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:24.135334 25437 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:24.135578 25507 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.135586 25437 raft_consensus.cc:3055] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:24.135689 25437 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:24.135914 25437 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:24.136242 25437 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.137897 25507 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 3.
I20251210 21:05:24.138264 25328 leader_election.cc:304] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:24.138489 25977 raft_consensus.cc:2804] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:24.138612 25977 raft_consensus.cc:697] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:24.138813 25977 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } }
I20251210 21:05:24.139667 25437 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 3.
I20251210 21:05:24.141956 25259 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 2 to 3, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 3 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:24.159844 26083 raft_consensus.cc:993] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower dbc0c6bd96c441e28d5a340d85fbe323 to start an election
I20251210 21:05:24.160122 25977 raft_consensus.cc:1081] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Signalling peer dbc0c6bd96c441e28d5a340d85fbe323 to start an election
I20251210 21:05:24.160725 25749 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee"
dest_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
from {username='slave'} at 127.0.0.1:46960
I20251210 21:05:24.160923 25749 raft_consensus.cc:493] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:24.161022 25749 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:24.163298 25749 raft_consensus.cc:515] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:24.164094 25749 leader_election.cc:290] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [CANDIDATE]: Term 4 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:24.164350 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:24.164510 25364 raft_consensus.cc:3055] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Stepping down as leader of term 3
I20251210 21:05:24.164577 25364 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:24.164803 25437 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f1698d0d3bd4765bc0897771cb0ceee" candidate_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:24.164829 25364 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:24.165104 25437 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:24.165189 25364 raft_consensus.cc:3060] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:24.166989 25437 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc0c6bd96c441e28d5a340d85fbe323 in term 4.
I20251210 21:05:24.166994 25364 raft_consensus.cc:2468] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc0c6bd96c441e28d5a340d85fbe323 in term 4.
I20251210 21:05:24.167428 25715 leader_election.cc:304] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb, dbc0c6bd96c441e28d5a340d85fbe323; no voters:
I20251210 21:05:24.167706 26079 raft_consensus.cc:2804] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 FOLLOWER]: Leader election won for term 4
I20251210 21:05:24.168181 26079 raft_consensus.cc:697] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Becoming Leader. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Running, Role: LEADER
I20251210 21:05:24.168428 26079 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:24.170548 25259 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: term changed from 3 to 4, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68). New cstate: current_term: 4 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:24.185786 25364 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 28. Preceding OpId from leader: term: 5 index: 29. (index mismatch)
I20251210 21:05:24.186329 26104 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:24.190603 25437 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 28. Preceding OpId from leader: term: 5 index: 29. (index mismatch)
I20251210 21:05:24.191227 26104 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:24.197293 25993 raft_consensus.cc:993] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:24.197438 26096 raft_consensus.cc:1081] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:24.197986 25364 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:24.198135 25364 raft_consensus.cc:493] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:24.198227 25364 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:24.200573 25364 raft_consensus.cc:515] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:24.201133 25364 leader_election.cc:290] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:24.201444 25436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:24.201579 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:24.201648 25436 raft_consensus.cc:3055] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:24.201735 25436 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:24.201748 25507 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:24.201969 25436 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:24.202311 25436 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:24.204164 25507 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:24.204489 25328 leader_election.cc:304] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:24.204723 25977 raft_consensus.cc:2804] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:24.204852 25977 raft_consensus.cc:697] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:24.205152 25436 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 2.
I20251210 21:05:24.205133 25977 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:24.208284 25259 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 1 to 2, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 2 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:24.214373 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:24.214550 25364 raft_consensus.cc:606] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Received request to transfer leadership
I20251210 21:05:24.216581 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:24.216732 25364 raft_consensus.cc:606] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Received request to transfer leadership
I20251210 21:05:24.217617 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:24.217772 25364 raft_consensus.cc:606] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Received request to transfer leadership
W20251210 21:05:24.218230 25311 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for e7e5750d29e94bad943f054f32b40746 already in progress
W20251210 21:05:24.218289 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for e7e5750d29e94bad943f054f32b40746 already in progress
I20251210 21:05:24.476315 25437 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:24.476886 26104 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:24.479583 26045 raft_consensus.cc:993] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:24.479736 26045 raft_consensus.cc:1081] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:24.480732 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43842
I20251210 21:05:24.480924 25435 raft_consensus.cc:493] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:24.481030 25435 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:24.481135 25364 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:24.481714 26045 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:24.484467 25435 raft_consensus.cc:515] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:24.485435 25437 raft_consensus.cc:1240] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Rejecting Update request from peer 56e19ce44c0c4f08b1eb440e1ef3e9cd for earlier term 3. Current term is 4. Ops: []
I20251210 21:05:24.486065 25435 leader_election.cc:290] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 4 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:24.486111 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:24.486274 25364 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:24.486435 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:24.486554 25507 raft_consensus.cc:3055] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Stepping down as leader of term 3
I20251210 21:05:24.486608 25507 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:24.486867 25507 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:24.487180 25507 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:24.489756 25364 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 4.
I20251210 21:05:24.489795 25899 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEARNER]: Advancing to term 3
I20251210 21:05:24.489982 25507 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 4.
I20251210 21:05:24.490365 25403 leader_election.cc:304] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:24.490674 25993 raft_consensus.cc:2804] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Leader election won for term 4
I20251210 21:05:24.490861 25993 raft_consensus.cc:697] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:24.491142 25993 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 3.29, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:24.492453 25899 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 LEARNER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:24.493913 25258 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 3 to 4, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 4 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:24.568940 25364 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:24.569574 26079 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:24.574482 25435 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:24.575023 26127 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:24.614324 25507 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251210 21:05:24.614864 25977 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:24.618254 25435 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251210 21:05:24.619122 25977 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:24.680184 25435 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:24.680718 25977 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:24.683068 26083 raft_consensus.cc:993] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:24.683290 25977 raft_consensus.cc:1081] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:24.683923 25437 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43858
I20251210 21:05:24.684149 25437 raft_consensus.cc:493] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:24.684257 25437 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.685050 25507 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:24.687263 25437 raft_consensus.cc:515] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:24.688473 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:24.688616 25363 raft_consensus.cc:3055] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:24.688681 25363 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:24.688869 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:24.688932 25363 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:24.689051 25507 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.689132 25363 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:24.691195 25363 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 3.
I20251210 21:05:24.688334 25437 leader_election.cc:290] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 3 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:24.691630 25405 leader_election.cc:304] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:24.691820 25435 raft_consensus.cc:1240] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Rejecting Update request from peer 128e4ab89fe449e8b5fdc68590b7ac89 for earlier term 2. Current term is 3. Ops: []
I20251210 21:05:24.691978 25993 raft_consensus.cc:2804] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:24.692157 25993 raft_consensus.cc:697] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:24.692190 25507 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 3.
I20251210 21:05:24.692488 25993 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 2.27, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:24.695281 25259 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 2 to 3, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 3 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:24.704257 25825 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 1 LEARNER]: Advancing to term 2
I20251210 21:05:24.706362 25825 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 2 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251210 21:05:24.885898 25363 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251210 21:05:24.886523 25957 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:24.889818 25899 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 LEARNER]: Advancing to term 4
I20251210 21:05:24.890012 25507 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251210 21:05:24.890455 25957 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:24.892833 25899 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251210 21:05:24.894284 25993 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:24.897400 25993 raft_consensus.cc:1064] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: attempting to promote NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 to VOTER
I20251210 21:05:24.898023 25993 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 4.30, Last appended by leader: 29, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:24.900233 25363 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:24.900247 25507 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:24.900485 25899 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:24.900801 26139 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:24.901098 26096 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:24.901290 25993 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:24.903478 26139 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.904062 25363 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.907184 25507 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.908119 25259 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 28 to 31, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:24.907176 25899 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.911880 25435 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:24.913691 25363 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:24.914266 26096 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:24.914362 25898 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:24.915351 26096 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:24.917068 26139 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.917650 25364 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.920899 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 05b90891884a4db3bc8723b604cec5ea with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:24.921345 25898 raft_consensus.cc:2955] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:24.921469 25259 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New cstate: current_term: 4 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:24.925719 25499 tablet_service.cc:1552] Processing DeleteTablet for tablet 05b90891884a4db3bc8723b604cec5ea with delete_type TABLET_DATA_TOMBSTONED (TS 56e19ce44c0c4f08b1eb440e1ef3e9cd not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:24.926532 26140 tablet_replica.cc:333] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:24.926720 26140 raft_consensus.cc:2243] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:24.927021 26140 raft_consensus.cc:2272] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:24.927825 26140 ts_tablet_manager.cc:1916] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:24.932901 26140 ts_tablet_manager.cc:1929] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.31
I20251210 21:05:24.932997 26140 log.cc:1199] T 05b90891884a4db3bc8723b604cec5ea P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/05b90891884a4db3bc8723b604cec5ea
I20251210 21:05:24.933568 25247 catalog_manager.cc:4985] TS 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213): tablet 05b90891884a4db3bc8723b604cec5ea (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:25.144243 25507 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251210 21:05:25.144749 26096 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:25.148665 25825 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 2 LEARNER]: Advancing to term 3
I20251210 21:05:25.148923 25364 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251210 21:05:25.149396 26139 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:25.151610 25825 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251210 21:05:25.152474 25993 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 26, Time since last communication: 0.000s
I20251210 21:05:25.155488 25993 raft_consensus.cc:1064] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: attempting to promote NON_VOTER a6abef32d32c46478bc3f2b9aa589efb to VOTER
I20251210 21:05:25.156275 25993 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 3.28, Last appended by leader: 27, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:25.158180 25825 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:25.158249 25364 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:25.158329 25507 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:25.158723 25993 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:25.159044 26096 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:25.159379 25993 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:25.160897 26096 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Committing config change with OpId 3.29: config changed from index 26 to 29, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.161953 25825 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index 26 to 29, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.164150 25507 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index 26 to 29, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.164707 25364 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index 26 to 29, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.166181 25259 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 26 to 29, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:25.170993 25435 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:25.172629 25507 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:25.172643 25825 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:25.173038 26139 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:25.173537 25993 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:25.175773 25957 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Committing config change with OpId 3.30: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.176398 25507 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.176419 25825 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.179991 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet e7e5750d29e94bad943f054f32b40746 with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:25.180640 25259 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New cstate: current_term: 3 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:25.184020 25352 tablet_service.cc:1552] Processing DeleteTablet for tablet e7e5750d29e94bad943f054f32b40746 with delete_type TABLET_DATA_TOMBSTONED (TS 128e4ab89fe449e8b5fdc68590b7ac89 not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:25.184840 26144 tablet_replica.cc:333] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:25.185058 26144 raft_consensus.cc:2243] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:25.185529 26144 raft_consensus.cc:2272] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:25.186430 26144 ts_tablet_manager.cc:1916] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:25.190793 26144 ts_tablet_manager.cc:1929] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.29
I20251210 21:05:25.190891 26144 log.cc:1199] T e7e5750d29e94bad943f054f32b40746 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/e7e5750d29e94bad943f054f32b40746
I20251210 21:05:25.191507 25249 catalog_manager.cc:4985] TS 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): tablet e7e5750d29e94bad943f054f32b40746 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:25.224267 25507 consensus_queue.cc:237] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 3.28, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:25.226655 25364 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:25.227149 25435 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:25.227229 26146 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:25.227648 26106 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:25.229996 26106 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:25.230998 25435 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:25.232025 25364 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
W20251210 21:05:25.233996 25477 consensus_peers.cc:597] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22e5c53fdb9d496b8b87aa938e9a4c3f. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:25.235766 25256 catalog_manager.cc:5654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New cstate: current_term: 3 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:25.247058 25825 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:25.249728 25435 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:25.249722 25507 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:25.250406 26153 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:25.250711 26053 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:25.253207 26053 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index 29 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:25.253770 25435 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 29 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
W20251210 21:05:25.257812 25791 consensus_peers.cc:597] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb -> Peer 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549): Couldn't send request to peer 90c04d3cdd4b4cb18b2a265365b1b094. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 681d997613474450ba6b9ee3de0f3f24. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:25.257584 25259 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb reported cstate change: config changed from index 29 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New cstate: current_term: 3 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:25.259732 25507 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 29 to 31, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:25.260644 25435 consensus_queue.cc:237] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:25.262817 25364 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:25.263365 26139 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20251210 21:05:25.263741 25406 consensus_peers.cc:597] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b65abf7cc0484fcda919a741e23928a7. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:25.264681 25507 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:25.265317 25993 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:25.265954 25957 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:25.266546 25364 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:25.268878 25507 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:25.271031 25257 catalog_manager.cc:5654] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:25.271982 25509 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 5.29, Last appended by leader: 28, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:25.274398 25364 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 29. Preceding OpId from leader: term: 5 index: 30. (index mismatch)
I20251210 21:05:25.274580 25435 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 29. Preceding OpId from leader: term: 5 index: 30. (index mismatch)
I20251210 21:05:25.275033 26106 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:25.276357 26146 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
W20251210 21:05:25.276623 25477 consensus_peers.cc:597] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 544270c4ded24ee2ae32f6aa4ec505c2. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:25.277626 26106 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 LEADER]: Committing config change with OpId 5.30: config changed from index -1 to 30, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:25.279644 25435 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Committing config change with OpId 5.30: config changed from index -1 to 30, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:25.280230 25364 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Committing config change with OpId 5.30: config changed from index -1 to 30, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:25.282261 25257 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index -1 to 30, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New cstate: current_term: 5 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:25.282397 25363 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:25.285048 25509 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:25.285097 25435 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251210 21:05:25.285559 26132 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20251210 21:05:25.285799 25331 consensus_peers.cc:597] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4810894f161642aba143677e92893e72. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:25.286124 26132 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251210 21:05:25.289026 26132 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:25.289844 25509 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:25.291896 25435 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:25.294657 25259 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index -1 to 28, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:25.303234 25749 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:25.305186 25435 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251210 21:05:25.305518 25364 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251210 21:05:25.305632 26114 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:25.305958 26127 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:25.308128 26079 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 31 to 33, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:25.308634 25364 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 31 to 33, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:25.312978 25435 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 31 to 33, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:25.313316 25257 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 31 to 33, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New cstate: current_term: 4 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251210 21:05:25.315973 25716 consensus_peers.cc:597] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 -> Peer 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549): Couldn't send request to peer 90c04d3cdd4b4cb18b2a265365b1b094. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f1698d0d3bd4765bc0897771cb0ceee. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:25.676025 26170 ts_tablet_manager.cc:933] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: Initiating tablet copy from peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:25.676800 26170 tablet_copy_client.cc:323] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:41411
I20251210 21:05:25.677399 25375 tablet_copy_service.cc:140] P 128e4ab89fe449e8b5fdc68590b7ac89: Received BeginTabletCopySession request for tablet 4810894f161642aba143677e92893e72 from peer dbc0c6bd96c441e28d5a340d85fbe323 ({username='slave'} at 127.0.0.1:55598)
I20251210 21:05:25.677510 25375 tablet_copy_service.cc:161] P 128e4ab89fe449e8b5fdc68590b7ac89: Beginning new tablet copy session on tablet 4810894f161642aba143677e92893e72 from peer dbc0c6bd96c441e28d5a340d85fbe323 at {username='slave'} at 127.0.0.1:55598: session id = dbc0c6bd96c441e28d5a340d85fbe323-4810894f161642aba143677e92893e72
I20251210 21:05:25.679795 25375 tablet_copy_source_session.cc:215] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:25.681017 26170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4810894f161642aba143677e92893e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:25.685870 26170 tablet_copy_client.cc:806] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:25.686183 26170 tablet_copy_client.cc:670] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:25.687592 26171 tablet_copy_client.cc:1025] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:25.690135 26172 ts_tablet_manager.cc:933] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: Initiating tablet copy from peer 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:25.690873 26172 tablet_copy_client.cc:323] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:39981
I20251210 21:05:25.691426 25450 tablet_copy_service.cc:140] P 4af27de48bec4590bfcd8d71056814eb: Received BeginTabletCopySession request for tablet b65abf7cc0484fcda919a741e23928a7 from peer dbc0c6bd96c441e28d5a340d85fbe323 ({username='slave'} at 127.0.0.1:33316)
I20251210 21:05:25.691581 25450 tablet_copy_service.cc:161] P 4af27de48bec4590bfcd8d71056814eb: Beginning new tablet copy session on tablet b65abf7cc0484fcda919a741e23928a7 from peer dbc0c6bd96c441e28d5a340d85fbe323 at {username='slave'} at 127.0.0.1:33316: session id = dbc0c6bd96c441e28d5a340d85fbe323-b65abf7cc0484fcda919a741e23928a7
I20251210 21:05:25.694150 25450 tablet_copy_source_session.cc:215] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:25.695694 26174 ts_tablet_manager.cc:933] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: Initiating tablet copy from peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845)
I20251210 21:05:25.695901 26172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65abf7cc0484fcda919a741e23928a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:25.696187 26175 ts_tablet_manager.cc:933] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: Initiating tablet copy from peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:25.698496 26175 tablet_copy_client.cc:323] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.69:45341
I20251210 21:05:25.704797 26172 tablet_copy_client.cc:806] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:25.705128 26172 tablet_copy_client.cc:670] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:25.706867 26174 tablet_copy_client.cc:323] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.68:37845
I20251210 21:05:25.707171 26173 tablet_copy_client.cc:1025] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:25.707245 26179 ts_tablet_manager.cc:933] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: Initiating tablet copy from peer 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:25.710562 26179 tablet_copy_client.cc:323] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:36213
I20251210 21:05:25.711213 25524 tablet_copy_service.cc:140] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Received BeginTabletCopySession request for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f from peer a6abef32d32c46478bc3f2b9aa589efb ({username='slave'} at 127.0.0.1:55886)
I20251210 21:05:25.711359 25524 tablet_copy_service.cc:161] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Beginning new tablet copy session on tablet 22e5c53fdb9d496b8b87aa938e9a4c3f from peer a6abef32d32c46478bc3f2b9aa589efb at {username='slave'} at 127.0.0.1:55886: session id = a6abef32d32c46478bc3f2b9aa589efb-22e5c53fdb9d496b8b87aa938e9a4c3f
I20251210 21:05:25.714335 25524 tablet_copy_source_session.cc:215] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:25.715618 26179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:25.716032 25760 tablet_copy_service.cc:140] P dbc0c6bd96c441e28d5a340d85fbe323: Received BeginTabletCopySession request for tablet 4f1698d0d3bd4765bc0897771cb0ceee from peer 90c04d3cdd4b4cb18b2a265365b1b094 ({username='slave'} at 127.0.0.1:46992)
I20251210 21:05:25.716228 25760 tablet_copy_service.cc:161] P dbc0c6bd96c441e28d5a340d85fbe323: Beginning new tablet copy session on tablet 4f1698d0d3bd4765bc0897771cb0ceee from peer 90c04d3cdd4b4cb18b2a265365b1b094 at {username='slave'} at 127.0.0.1:46992: session id = 90c04d3cdd4b4cb18b2a265365b1b094-4f1698d0d3bd4765bc0897771cb0ceee
I20251210 21:05:25.718096 25835 tablet_copy_service.cc:140] P a6abef32d32c46478bc3f2b9aa589efb: Received BeginTabletCopySession request for tablet 681d997613474450ba6b9ee3de0f3f24 from peer 90c04d3cdd4b4cb18b2a265365b1b094 ({username='slave'} at 127.0.0.1:56498)
I20251210 21:05:25.718266 25835 tablet_copy_service.cc:161] P a6abef32d32c46478bc3f2b9aa589efb: Beginning new tablet copy session on tablet 681d997613474450ba6b9ee3de0f3f24 from peer 90c04d3cdd4b4cb18b2a265365b1b094 at {username='slave'} at 127.0.0.1:56498: session id = 90c04d3cdd4b4cb18b2a265365b1b094-681d997613474450ba6b9ee3de0f3f24
I20251210 21:05:25.719535 25760 tablet_copy_source_session.cc:215] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:25.720707 26174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f1698d0d3bd4765bc0897771cb0ceee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:25.720757 25835 tablet_copy_source_session.cc:215] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:25.721848 26175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681d997613474450ba6b9ee3de0f3f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:25.725957 26174 tablet_copy_client.cc:806] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:25.726207 26174 tablet_copy_client.cc:670] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:25.727953 26176 tablet_copy_client.cc:1025] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:25.728370 26175 tablet_copy_client.cc:806] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:25.728572 26175 tablet_copy_client.cc:670] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:25.728935 26179 tablet_copy_client.cc:806] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:25.729139 26179 tablet_copy_client.cc:670] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:25.730127 26177 tablet_copy_client.cc:1025] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:25.730728 26181 tablet_copy_client.cc:1025] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:25.747057 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:25.750522 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:25.750730 25435 raft_consensus.cc:606] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:25.751682 25509 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:25.751869 25509 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:25.752902 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "90c04d3cdd4b4cb18b2a265365b1b094"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:25.753057 25364 raft_consensus.cc:606] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Received request to transfer leadership to TS 90c04d3cdd4b4cb18b2a265365b1b094
I20251210 21:05:25.753912 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:25.754036 25435 raft_consensus.cc:606] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Received request to transfer leadership to TS dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:25.754346 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20251210 21:05:25.754558 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:25.769315 26182 ts_tablet_manager.cc:933] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: Initiating tablet copy from peer 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:25.769932 26182 tablet_copy_client.cc:323] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:36213
I20251210 21:05:25.770489 25524 tablet_copy_service.cc:140] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Received BeginTabletCopySession request for tablet 544270c4ded24ee2ae32f6aa4ec505c2 from peer a6abef32d32c46478bc3f2b9aa589efb ({username='slave'} at 127.0.0.1:55886)
I20251210 21:05:25.770608 25524 tablet_copy_service.cc:161] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Beginning new tablet copy session on tablet 544270c4ded24ee2ae32f6aa4ec505c2 from peer a6abef32d32c46478bc3f2b9aa589efb at {username='slave'} at 127.0.0.1:55886: session id = a6abef32d32c46478bc3f2b9aa589efb-544270c4ded24ee2ae32f6aa4ec505c2
I20251210 21:05:25.772840 25524 tablet_copy_source_session.cc:215] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:25.773874 26182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544270c4ded24ee2ae32f6aa4ec505c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:25.779124 26182 tablet_copy_client.cc:806] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:25.779358 26182 tablet_copy_client.cc:670] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:25.780937 26183 tablet_copy_client.cc:1025] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:25.902757 26096 raft_consensus.cc:993] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower dbc0c6bd96c441e28d5a340d85fbe323 to start an election
I20251210 21:05:25.902953 26096 raft_consensus.cc:1081] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Signalling peer dbc0c6bd96c441e28d5a340d85fbe323 to start an election
I20251210 21:05:25.903548 25750 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23"
dest_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
from {username='slave'} at 127.0.0.1:51110
I20251210 21:05:25.903723 25750 raft_consensus.cc:493] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:25.903927 25750 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:25.906226 25750 raft_consensus.cc:515] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:25.906847 25750 leader_election.cc:290] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [CANDIDATE]: Term 3 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549)
I20251210 21:05:25.907136 25437 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:25.907274 25437 raft_consensus.cc:3055] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:25.907229 25899 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7537ffe9f943b3a97e77adab5a1e23" candidate_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "90c04d3cdd4b4cb18b2a265365b1b094"
I20251210 21:05:25.907341 25437 raft_consensus.cc:740] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:25.907416 25899 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:25.907680 25437 consensus_queue.cc:260] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:25.907948 25437 raft_consensus.cc:3060] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:25.909637 25899 raft_consensus.cc:2468] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc0c6bd96c441e28d5a340d85fbe323 in term 3.
I20251210 21:05:25.909929 25437 raft_consensus.cc:2468] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc0c6bd96c441e28d5a340d85fbe323 in term 3.
I20251210 21:05:25.910017 25716 leader_election.cc:304] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c04d3cdd4b4cb18b2a265365b1b094, dbc0c6bd96c441e28d5a340d85fbe323; no voters:
I20251210 21:05:25.910260 26127 raft_consensus.cc:2804] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:25.910661 26127 raft_consensus.cc:697] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 LEADER]: Becoming Leader. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Running, Role: LEADER
I20251210 21:05:25.910953 26127 consensus_queue.cc:237] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:25.913977 25259 catalog_manager.cc:5654] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: term changed from 2 to 3, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68). New cstate: current_term: 3 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:25.964778 26096 raft_consensus.cc:993] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:25.964934 26096 raft_consensus.cc:1081] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:25.965509 25364 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:25.965709 25364 raft_consensus.cc:493] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:25.965890 25364 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:25.968271 25364 raft_consensus.cc:515] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:25.969302 25364 leader_election.cc:290] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 5 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549)
I20251210 21:05:25.969518 25898 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "90c04d3cdd4b4cb18b2a265365b1b094"
I20251210 21:05:25.969537 25437 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:25.969693 25898 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:25.969735 25437 raft_consensus.cc:3055] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Stepping down as leader of term 4
I20251210 21:05:25.969805 25437 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:25.970171 25437 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:25.970402 25437 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:25.972064 25898 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 5.
I20251210 21:05:25.972442 25331 leader_election.cc:304] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 90c04d3cdd4b4cb18b2a265365b1b094; no voters:
I20251210 21:05:25.972687 26132 raft_consensus.cc:2804] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Leader election won for term 5
I20251210 21:05:25.972766 25437 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 5.
I20251210 21:05:25.983645 26132 raft_consensus.cc:697] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:25.983914 26132 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:25.986445 25259 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 4 to 5, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 5 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:26.125692 26132 raft_consensus.cc:993] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 90c04d3cdd4b4cb18b2a265365b1b094 to start an election
I20251210 21:05:26.125849 26132 raft_consensus.cc:1081] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Signalling peer 90c04d3cdd4b4cb18b2a265365b1b094 to start an election
I20251210 21:05:26.126351 25898 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86"
dest_uuid: "90c04d3cdd4b4cb18b2a265365b1b094"
from {username='slave'} at 127.0.0.1:58702
I20251210 21:05:26.126515 25898 raft_consensus.cc:493] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:26.126607 25898 raft_consensus.cc:3060] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:26.128744 25898 raft_consensus.cc:515] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:26.129499 25898 leader_election.cc:290] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [CANDIDATE]: Term 2 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:26.129978 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:26.130160 25364 raft_consensus.cc:3055] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:26.130226 25364 raft_consensus.cc:740] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:26.130445 25364 consensus_queue.cc:260] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:26.130765 25364 raft_consensus.cc:3060] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:26.132922 25364 raft_consensus.cc:2468] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c04d3cdd4b4cb18b2a265365b1b094 in term 2.
I20251210 21:05:26.133306 25864 leader_election.cc:304] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 90c04d3cdd4b4cb18b2a265365b1b094; no voters:
I20251210 21:05:26.133839 26192 raft_consensus.cc:2804] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:26.135843 26192 raft_consensus.cc:697] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEADER]: Becoming Leader. State: Replica: 90c04d3cdd4b4cb18b2a265365b1b094, State: Running, Role: LEADER
I20251210 21:05:26.136281 26192 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:26.136762 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6e20042b566343a7acdbe06f4dcb0c86" candidate_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:26.136930 25509 raft_consensus.cc:3060] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:26.139036 25509 raft_consensus.cc:2468] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c04d3cdd4b4cb18b2a265365b1b094 in term 2.
I20251210 21:05:26.139750 25259 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 reported cstate change: term changed from 1 to 2, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70). New cstate: current_term: 2 leader_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:26.269130 26195 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:26.269280 26195 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:26.269840 25437 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43842
I20251210 21:05:26.270120 25437 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:26.270258 25437 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:26.272970 25437 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:26.273475 25437 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 6 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:26.273941 25364 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 30 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:26.273948 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 30 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:26.274132 25509 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 LEADER]: Stepping down as leader of term 5
I20251210 21:05:26.274117 25364 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:26.274235 25509 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:26.274477 25509 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 5.30, Last appended by leader: 30, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:26.274847 25509 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:26.276939 25364 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 6.
I20251210 21:05:26.277314 25405 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:26.277570 25957 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Leader election won for term 6
I20251210 21:05:26.277695 25957 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:26.277956 25509 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 6.
I20251210 21:05:26.277917 25957 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 5.30, Last appended by leader: 30, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:26.280347 25259 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 5 to 6, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 6 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:26.286130 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:26.286295 25435 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Received request to transfer leadership
I20251210 21:05:26.293514 25437 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:26.293670 25437 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Received request to transfer leadership
I20251210 21:05:26.294721 25437 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:26.294860 25437 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Received request to transfer leadership
W20251210 21:05:26.295301 25311 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 544270c4ded24ee2ae32f6aa4ec505c2 already in progress
W20251210 21:05:26.295372 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 544270c4ded24ee2ae32f6aa4ec505c2 already in progress
I20251210 21:05:26.359614 25437 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20251210 21:05:26.360306 26202 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:26.380188 25435 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251210 21:05:26.380738 26197 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:26.384662 25898 raft_consensus.cc:1275] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20251210 21:05:26.385391 26127 consensus_queue.cc:1048] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:26.388520 25899 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 5 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251210 21:05:26.389066 26197 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:26.527724 25507 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251210 21:05:26.528231 26192 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:26.531523 25363 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251210 21:05:26.531984 26192 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:26.679548 25363 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 30. Preceding OpId from leader: term: 6 index: 31. (index mismatch)
I20251210 21:05:26.680024 26198 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:26.682924 25957 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:26.683070 25957 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:26.683655 25364 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:26.684270 25364 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:26.684386 25364 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Advancing to term 7
I20251210 21:05:26.686529 25364 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:26.687100 25364 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 7 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:26.687602 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 7 candidate_status { last_received { term: 6 index: 31 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:26.687743 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 7 candidate_status { last_received { term: 6 index: 31 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:26.687908 25435 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Stepping down as leader of term 6
I20251210 21:05:26.687763 25509 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 6 FOLLOWER]: Advancing to term 7
W20251210 21:05:26.688650 25403 consensus_peers.cc:597] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:26.690197 25509 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 7.
I20251210 21:05:26.687968 25435 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:26.690625 25328 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:26.690743 25435 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 6.31, Last appended by leader: 31, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:26.690881 26197 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 FOLLOWER]: Leader election won for term 7
I20251210 21:05:26.691022 26197 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:26.691113 25435 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Advancing to term 7
I20251210 21:05:26.691241 26197 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 6.31, Last appended by leader: 31, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:26.693245 25435 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 7.
I20251210 21:05:26.694087 25507 raft_consensus.cc:1240] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Rejecting Update request from peer 4af27de48bec4590bfcd8d71056814eb for earlier term 6. Current term is 7. Ops: []
I20251210 21:05:26.694928 25257 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 6 to 7, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 7 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:27.072822 25435 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 6 index: 31. Preceding OpId from leader: term: 7 index: 32. (index mismatch)
I20251210 21:05:27.073431 26197 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
W20251210 21:05:27.077497 25328 consensus_peers.cc:597] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:27.078315 25507 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 5 index: 30. Preceding OpId from leader: term: 7 index: 32. (index mismatch)
I20251210 21:05:27.078822 26132 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:27.687942 26170 tablet_copy_client.cc:538] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:27.691154 26170 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap starting.
I20251210 21:05:27.707603 26172 tablet_copy_client.cc:538] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:27.710222 26172 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap starting.
I20251210 21:05:27.728281 26174 tablet_copy_client.cc:538] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:27.729373 26170 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=106 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:27.729674 26170 tablet_bootstrap.cc:492] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap complete.
I20251210 21:05:27.729918 26170 ts_tablet_manager.cc:1403] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent bootstrapping tablet: real 0.039s user 0.034s sys 0.004s
I20251210 21:05:27.730476 26175 tablet_copy_client.cc:538] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:27.730697 26170 raft_consensus.cc:359] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:27.730875 26170 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Initialized, Role: LEARNER
I20251210 21:05:27.730957 26174 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap starting.
I20251210 21:05:27.731051 26170 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:27.731575 26170 ts_tablet_manager.cc:1434] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251210 21:05:27.732506 25375 tablet_copy_service.cc:342] P 128e4ab89fe449e8b5fdc68590b7ac89: Request end of tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-4810894f161642aba143677e92893e72 received from {username='slave'} at 127.0.0.1:55598
I20251210 21:05:27.732606 25375 tablet_copy_service.cc:434] P 128e4ab89fe449e8b5fdc68590b7ac89: ending tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-4810894f161642aba143677e92893e72 on tablet 4810894f161642aba143677e92893e72 with peer dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:27.733925 26175 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap starting.
I20251210 21:05:27.735631 26179 tablet_copy_client.cc:538] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:27.739162 26179 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap starting.
I20251210 21:05:27.754926 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:27.759240 25364 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:27.759490 25364 raft_consensus.cc:606] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:27.760602 25363 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:27.760761 25363 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Received request to transfer leadership to TS 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:27.761121 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251210 21:05:27.761440 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:27.771363 26172 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:27.772199 26172 tablet_bootstrap.cc:492] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap complete.
I20251210 21:05:27.772806 26172 ts_tablet_manager.cc:1403] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent bootstrapping tablet: real 0.063s user 0.034s sys 0.005s
I20251210 21:05:27.774398 26172 raft_consensus.cc:359] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:27.774978 26172 raft_consensus.cc:740] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Initialized, Role: LEARNER
I20251210 21:05:27.775218 26172 consensus_queue.cc:260] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:27.776155 26172 ts_tablet_manager.cc:1434] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251210 21:05:27.776871 25450 tablet_copy_service.cc:342] P 4af27de48bec4590bfcd8d71056814eb: Request end of tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-b65abf7cc0484fcda919a741e23928a7 received from {username='slave'} at 127.0.0.1:33316
I20251210 21:05:27.776983 25450 tablet_copy_service.cc:434] P 4af27de48bec4590bfcd8d71056814eb: ending tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-b65abf7cc0484fcda919a741e23928a7 on tablet b65abf7cc0484fcda919a741e23928a7 with peer dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:27.778748 26174 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=89 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:27.779194 26174 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap complete.
I20251210 21:05:27.779558 26174 ts_tablet_manager.cc:1403] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent bootstrapping tablet: real 0.049s user 0.045s sys 0.002s
I20251210 21:05:27.780496 26174 raft_consensus.cc:359] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:27.780848 26174 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 90c04d3cdd4b4cb18b2a265365b1b094, State: Initialized, Role: LEARNER
I20251210 21:05:27.781106 26174 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:27.781663 26182 tablet_copy_client.cc:538] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:27.781690 26174 ts_tablet_manager.cc:1434] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:05:27.782274 25760 tablet_copy_service.cc:342] P dbc0c6bd96c441e28d5a340d85fbe323: Request end of tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-4f1698d0d3bd4765bc0897771cb0ceee received from {username='slave'} at 127.0.0.1:46992
I20251210 21:05:27.782451 25760 tablet_copy_service.cc:434] P dbc0c6bd96c441e28d5a340d85fbe323: ending tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-4f1698d0d3bd4765bc0897771cb0ceee on tablet 4f1698d0d3bd4765bc0897771cb0ceee with peer 90c04d3cdd4b4cb18b2a265365b1b094
I20251210 21:05:27.784631 26182 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap starting.
I20251210 21:05:27.787413 26224 raft_consensus.cc:993] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:27.787578 26224 raft_consensus.cc:1081] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:27.788228 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43858
I20251210 21:05:27.788424 25435 raft_consensus.cc:493] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:27.788537 25435 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:27.790495 25435 raft_consensus.cc:515] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:27.791054 25435 leader_election.cc:290] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 6 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549)
I20251210 21:05:27.791329 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:27.791543 25363 raft_consensus.cc:3055] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Stepping down as leader of term 5
I20251210 21:05:27.791631 25363 raft_consensus.cc:740] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:27.791893 25363 consensus_queue.cc:260] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:27.791978 25898 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "05b90891884a4db3bc8723b604cec5ea" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "90c04d3cdd4b4cb18b2a265365b1b094"
I20251210 21:05:27.792205 25898 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:27.794760 25898 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 6.
I20251210 21:05:27.792212 25363 raft_consensus.cc:3060] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:27.797894 25363 raft_consensus.cc:2468] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 6.
I20251210 21:05:27.791963 26175 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=102 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:27.798691 26175 tablet_bootstrap.cc:492] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap complete.
I20251210 21:05:27.798282 25405 leader_election.cc:304] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:27.798975 26175 ts_tablet_manager.cc:1403] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent bootstrapping tablet: real 0.065s user 0.043s sys 0.000s
I20251210 21:05:27.799141 26198 raft_consensus.cc:2804] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Leader election won for term 6
I20251210 21:05:27.799762 26175 raft_consensus.cc:359] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:27.799988 26175 raft_consensus.cc:740] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 90c04d3cdd4b4cb18b2a265365b1b094, State: Initialized, Role: LEARNER
I20251210 21:05:27.800240 26175 consensus_queue.cc:260] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:27.800930 26175 ts_tablet_manager.cc:1434] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:05:27.802867 25835 tablet_copy_service.cc:342] P a6abef32d32c46478bc3f2b9aa589efb: Request end of tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-681d997613474450ba6b9ee3de0f3f24 received from {username='slave'} at 127.0.0.1:56498
I20251210 21:05:27.803001 25835 tablet_copy_service.cc:434] P a6abef32d32c46478bc3f2b9aa589efb: ending tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-681d997613474450ba6b9ee3de0f3f24 on tablet 681d997613474450ba6b9ee3de0f3f24 with peer 90c04d3cdd4b4cb18b2a265365b1b094
I20251210 21:05:27.812366 26179 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=106 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:27.812906 26179 tablet_bootstrap.cc:492] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap complete.
I20251210 21:05:27.813280 26179 ts_tablet_manager.cc:1403] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: Time spent bootstrapping tablet: real 0.074s user 0.047s sys 0.006s
I20251210 21:05:27.814215 26179 raft_consensus.cc:359] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:27.814482 26179 raft_consensus.cc:740] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Initialized, Role: LEARNER
I20251210 21:05:27.814774 26179 consensus_queue.cc:260] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:27.815536 26179 ts_tablet_manager.cc:1434] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:27.816426 25524 tablet_copy_service.cc:342] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Request end of tablet copy session a6abef32d32c46478bc3f2b9aa589efb-22e5c53fdb9d496b8b87aa938e9a4c3f received from {username='slave'} at 127.0.0.1:55886
I20251210 21:05:27.816574 25524 tablet_copy_service.cc:434] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: ending tablet copy session a6abef32d32c46478bc3f2b9aa589efb-22e5c53fdb9d496b8b87aa938e9a4c3f on tablet 22e5c53fdb9d496b8b87aa938e9a4c3f with peer a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:27.820349 26198 raft_consensus.cc:697] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:27.820652 26198 consensus_queue.cc:237] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:27.825604 25257 catalog_manager.cc:5654] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 5 to 6, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 6 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:27.849820 26182 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=88 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:27.850232 26182 tablet_bootstrap.cc:492] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap complete.
I20251210 21:05:27.850505 26182 ts_tablet_manager.cc:1403] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: Time spent bootstrapping tablet: real 0.066s user 0.032s sys 0.005s
I20251210 21:05:27.851186 26182 raft_consensus.cc:359] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:27.851361 26182 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Initialized, Role: LEARNER
I20251210 21:05:27.851846 26182 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 5.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:27.852408 26182 ts_tablet_manager.cc:1434] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251210 21:05:27.853139 25524 tablet_copy_service.cc:342] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Request end of tablet copy session a6abef32d32c46478bc3f2b9aa589efb-544270c4ded24ee2ae32f6aa4ec505c2 received from {username='slave'} at 127.0.0.1:55886
I20251210 21:05:27.853278 25524 tablet_copy_service.cc:434] P 56e19ce44c0c4f08b1eb440e1ef3e9cd: ending tablet copy session a6abef32d32c46478bc3f2b9aa589efb-544270c4ded24ee2ae32f6aa4ec505c2 on tablet 544270c4ded24ee2ae32f6aa4ec505c2 with peer a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:27.853994 25898 raft_consensus.cc:1217] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.30->[3.31-3.31] Dedup: 3.31->[]
I20251210 21:05:28.017958 25750 raft_consensus.cc:1217] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.27->[2.28-2.28] Dedup: 2.28->[]
I20251210 21:05:28.030930 25750 raft_consensus.cc:1217] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.27->[2.28-2.28] Dedup: 2.28->[]
I20251210 21:05:28.061379 25825 raft_consensus.cc:1217] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Deduplicated request from leader. Original: 3.28->[3.29-3.29] Dedup: 3.29->[]
I20251210 21:05:28.142750 25898 raft_consensus.cc:1217] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.32->[4.33-4.33] Dedup: 4.33->[]
I20251210 21:05:28.199227 25825 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 5 LEARNER]: Advancing to term 7
I20251210 21:05:28.201664 25825 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 7 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 5 index: 30. Preceding OpId from leader: term: 6 index: 31. (index mismatch)
I20251210 21:05:28.202195 26226 consensus_queue.cc:1050] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:28.209365 26226 raft_consensus.cc:1064] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: attempting to promote NON_VOTER a6abef32d32c46478bc3f2b9aa589efb to VOTER
I20251210 21:05:28.210093 26226 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 7.32, Last appended by leader: 31, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:28.213303 25825 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 7 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 7 index: 32. Preceding OpId from leader: term: 7 index: 33. (index mismatch)
I20251210 21:05:28.214113 26224 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:28.214113 25507 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 7 index: 32. Preceding OpId from leader: term: 7 index: 33. (index mismatch)
I20251210 21:05:28.215256 26224 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:28.218256 26224 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 LEADER]: Committing config change with OpId 7.33: config changed from index 30 to 33, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.218715 25507 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Committing config change with OpId 7.33: config changed from index 30 to 33, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.223953 25259 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index 30 to 33, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.226357 25435 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 7 index: 32. Preceding OpId from leader: term: 7 index: 33. (index mismatch)
I20251210 21:05:28.228016 26226 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:28.228688 25825 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 7 FOLLOWER]: Committing config change with OpId 7.33: config changed from index 30 to 33, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.236354 25363 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 7.33, Last appended by leader: 31, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:28.237752 25435 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Committing config change with OpId 7.33: config changed from index 30 to 33, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.238866 25825 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 7 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 7 index: 33. Preceding OpId from leader: term: 7 index: 34. (index mismatch)
I20251210 21:05:28.239127 25507 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 7 index: 33. Preceding OpId from leader: term: 7 index: 34. (index mismatch)
I20251210 21:05:28.239475 26226 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:28.239871 26243 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:28.240725 25898 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 6 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251210 21:05:28.242018 26233 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:28.242993 26246 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 LEADER]: Committing config change with OpId 7.34: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.243989 25507 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Committing config change with OpId 7.34: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.246743 25249 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 544270c4ded24ee2ae32f6aa4ec505c2 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:28.247944 25257 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New cstate: current_term: 7 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:28.249063 25824 raft_consensus.cc:2955] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 7 FOLLOWER]: Committing config change with OpId 7.34: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.252410 25422 tablet_service.cc:1552] Processing DeleteTablet for tablet 544270c4ded24ee2ae32f6aa4ec505c2 with delete_type TABLET_DATA_TOMBSTONED (TS 4af27de48bec4590bfcd8d71056814eb not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:28.258087 26250 tablet_replica.cc:333] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:28.258428 26250 raft_consensus.cc:2243] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:28.258807 26250 raft_consensus.cc:2272] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:28.263639 26250 ts_tablet_manager.cc:1916] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:28.268563 25363 raft_consensus.cc:1275] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251210 21:05:28.269255 26233 consensus_queue.cc:1048] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:28.271901 26250 ts_tablet_manager.cc:1929] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.33
I20251210 21:05:28.272133 26250 log.cc:1199] T 544270c4ded24ee2ae32f6aa4ec505c2 P 4af27de48bec4590bfcd8d71056814eb: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/544270c4ded24ee2ae32f6aa4ec505c2
I20251210 21:05:28.273316 25247 catalog_manager.cc:4985] TS 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): tablet 544270c4ded24ee2ae32f6aa4ec505c2 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:28.292455 26224 raft_consensus.cc:993] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:28.292768 26224 raft_consensus.cc:1081] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Signalling peer 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:28.293305 25509 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:47102
I20251210 21:05:28.293512 25509 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:28.293628 25509 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:28.295863 25509 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:28.296623 25509 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981)
I20251210 21:05:28.296933 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:28.297103 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:28.297251 25363 raft_consensus.cc:3055] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:28.297356 25435 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:28.297397 25363 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:28.297720 25363 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:28.298134 25363 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:28.299870 25435 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:28.300355 25477 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, 56e19ce44c0c4f08b1eb440e1ef3e9cd; no voters:
I20251210 21:05:28.300611 26196 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:28.300997 26196 raft_consensus.cc:697] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:28.301272 26196 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:28.301656 25363 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:28.304729 25257 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 2 to 3, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 3 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:28.309723 25509 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:28.309928 25509 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Received request to transfer leadership
I20251210 21:05:28.315755 25509 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:28.315965 25509 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Received request to transfer leadership
W20251210 21:05:28.316277 25247 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 4810894f161642aba143677e92893e72 already in progress [suppressed 3 similar messages]
I20251210 21:05:28.317123 25509 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:28.317291 25509 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Received request to transfer leadership
W20251210 21:05:28.317667 25311 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 4810894f161642aba143677e92893e72 already in progress
W20251210 21:05:28.317739 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 4810894f161642aba143677e92893e72 already in progress
I20251210 21:05:28.421082 26259 raft_consensus.cc:1064] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: attempting to promote NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 to VOTER
I20251210 21:05:28.421955 26259 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 29, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:28.423996 25898 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 LEARNER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251210 21:05:28.424045 25507 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251210 21:05:28.424536 26261 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:28.424655 25435 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251210 21:05:28.425482 26258 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:28.425806 26261 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:28.428419 26258 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEADER]: Committing config change with OpId 3.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.429404 25507 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.433588 25898 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.433965 25257 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb reported cstate change: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:28.445228 25437 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 31 to 32, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.447721 25825 consensus_queue.cc:237] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:28.449496 25898 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251210 21:05:28.449882 25437 raft_consensus.cc:1275] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251210 21:05:28.449971 26258 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:28.451687 26233 raft_consensus.cc:1064] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
I20251210 21:05:28.452569 26233 consensus_queue.cc:237] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 26, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:28.454785 26258 consensus_queue.cc:1048] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:28.455193 25507 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:28.455332 26259 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEADER]: Committing config change with OpId 3.33: config changed from index 32 to 33, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.455847 25898 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 32 to 33, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.458480 25305 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 681d997613474450ba6b9ee3de0f3f24 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:28.458889 25364 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:28.459291 25749 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251210 21:05:28.459393 26198 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:28.459918 26198 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:28.460724 25259 catalog_manager.cc:5654] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb reported cstate change: config changed from index 32 to 33, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New cstate: current_term: 3 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:28.462834 25437 raft_consensus.cc:2955] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 32 to 33, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.463900 26198 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:28.467151 25499 tablet_service.cc:1552] Processing DeleteTablet for tablet 681d997613474450ba6b9ee3de0f3f24 with delete_type TABLET_DATA_TOMBSTONED (TS 56e19ce44c0c4f08b1eb440e1ef3e9cd not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:28.467931 26232 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.469427 25509 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.470089 26270 tablet_replica.cc:333] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:28.470281 26270 raft_consensus.cc:2243] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:28.470597 26270 raft_consensus.cc:2272] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:28.471546 26270 ts_tablet_manager.cc:1916] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:28.476660 25363 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.477475 25749 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.480901 26270 ts_tablet_manager.cc:1929] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.32
I20251210 21:05:28.481047 26270 log.cc:1199] T 681d997613474450ba6b9ee3de0f3f24 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/681d997613474450ba6b9ee3de0f3f24
I20251210 21:05:28.481014 25257 catalog_manager.cc:5654] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index 28 to 29, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.481767 25247 catalog_manager.cc:4985] TS 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213): tablet 681d997613474450ba6b9ee3de0f3f24 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:28.485110 26255 raft_consensus.cc:1064] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: attempting to promote NON_VOTER a6abef32d32c46478bc3f2b9aa589efb to VOTER
I20251210 21:05:28.485931 26255 consensus_queue.cc:237] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 27, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:28.486083 25437 consensus_queue.cc:237] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:28.488463 25509 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251210 21:05:28.488957 26232 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:28.489388 25825 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:28.489912 26196 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:28.490545 25435 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:28.491567 26198 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.491799 26196 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:28.492358 25509 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.492842 25749 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251210 21:05:28.493323 26233 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:28.494709 25750 raft_consensus.cc:2955] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:28.495079 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet b65abf7cc0484fcda919a741e23928a7 with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:28.496658 26196 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.498684 25258 catalog_manager.cc:5654] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 29 to 30, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New cstate: current_term: 2 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:28.500038 25437 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.503315 25352 tablet_service.cc:1552] Processing DeleteTablet for tablet b65abf7cc0484fcda919a741e23928a7 with delete_type TABLET_DATA_TOMBSTONED (TS 128e4ab89fe449e8b5fdc68590b7ac89 not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:28.505362 26276 tablet_replica.cc:333] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:28.505991 26276 raft_consensus.cc:2243] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:28.506417 26276 raft_consensus.cc:2272] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:28.506563 25256 catalog_manager.cc:5654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.507371 26276 ts_tablet_manager.cc:1916] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:28.509824 25364 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:28.510468 26272 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:28.513808 25825 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.517246 25364 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.519979 26276 ts_tablet_manager.cc:1929] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.29
I20251210 21:05:28.520128 26276 log.cc:1199] T b65abf7cc0484fcda919a741e23928a7 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/b65abf7cc0484fcda919a741e23928a7
I20251210 21:05:28.521344 25249 catalog_manager.cc:4985] TS 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): tablet b65abf7cc0484fcda919a741e23928a7 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:28.563830 26127 raft_consensus.cc:1064] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: attempting to promote NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 to VOTER
I20251210 21:05:28.564584 26127 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 31, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:28.567585 25437 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251210 21:05:28.568228 25364 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251210 21:05:28.568336 25898 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251210 21:05:28.568414 26281 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:28.568903 26127 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:28.569294 26280 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:28.585498 26127 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 33 to 34, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.586364 25364 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.590242 25437 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.591118 25256 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 33 to 34, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:28.592258 25898 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.596679 25749 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:28.599097 25898 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251210 21:05:28.599251 25437 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251210 21:05:28.600004 26280 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251210 21:05:28.600333 26127 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251210 21:05:28.602123 26281 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Committing config change with OpId 4.35: config changed from index 34 to 35, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.603039 25437 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 34 to 35, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.605098 25310 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 4f1698d0d3bd4765bc0897771cb0ceee with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:28.606827 25256 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 34 to 35, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New cstate: current_term: 4 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:28.607969 25898 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 34 to 35, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:28.610615 25352 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f1698d0d3bd4765bc0897771cb0ceee with delete_type TABLET_DATA_TOMBSTONED (TS 128e4ab89fe449e8b5fdc68590b7ac89 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:28.610972 26276 tablet_replica.cc:333] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:28.611174 26276 raft_consensus.cc:2243] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:28.611459 26276 raft_consensus.cc:2272] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:28.612936 26276 ts_tablet_manager.cc:1916] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:28.617388 26276 ts_tablet_manager.cc:1929] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.34
I20251210 21:05:28.617496 26276 log.cc:1199] T 4f1698d0d3bd4765bc0897771cb0ceee P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/4f1698d0d3bd4765bc0897771cb0ceee
I20251210 21:05:28.618116 25249 catalog_manager.cc:4985] TS 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): tablet 4f1698d0d3bd4765bc0897771cb0ceee (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:28.730986 25750 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Advancing to term 3
I20251210 21:05:28.733471 25750 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 LEARNER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:28.734084 26277 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:28.736738 26277 raft_consensus.cc:1064] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
W20251210 21:05:28.736886 26277 raft_consensus.cc:1068] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Unable to promote non-voter dbc0c6bd96c441e28d5a340d85fbe323: Service unavailable: leader transfer in progress
I20251210 21:05:28.755209 25364 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:28.755710 26277 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:28.757962 26196 raft_consensus.cc:993] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:28.758141 26196 raft_consensus.cc:1081] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:28.758790 25363 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39018
I20251210 21:05:28.758980 25363 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:28.759093 25363 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:28.761446 25363 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:28.762089 25363 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 4 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:28.762773 25437 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:28.762847 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:28.762980 25437 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:28.762980 25507 raft_consensus.cc:3055] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Stepping down as leader of term 3
I20251210 21:05:28.765537 25507 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:28.765762 25437 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 4.
I20251210 21:05:28.765812 25507 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:28.765988 25435 raft_consensus.cc:1240] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Rejecting Update request from peer 56e19ce44c0c4f08b1eb440e1ef3e9cd for earlier term 3. Current term is 4. Ops: []
I20251210 21:05:28.766183 25507 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:28.766786 25328 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:28.767076 26224 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Leader election won for term 4
I20251210 21:05:28.767259 26224 raft_consensus.cc:697] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:28.767556 26224 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:28.769317 25507 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 4.
I20251210 21:05:28.769831 26196 raft_consensus.cc:1064] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
W20251210 21:05:28.770195 26196 raft_consensus.cc:1068] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Unable to promote non-voter dbc0c6bd96c441e28d5a340d85fbe323: Illegal state: Replica 56e19ce44c0c4f08b1eb440e1ef3e9cd is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:28.770340 25258 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 3 to 4, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 4 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:28.941767 25507 consensus_queue.cc:237] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:28.943459 25825 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:28.943624 25363 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:28.944073 26272 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:28.944342 26275 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:28.946262 26272 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.947409 25825 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.949378 25247 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:28.949723 25363 raft_consensus.cc:2955] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:28.951602 25256 catalog_manager.cc:5654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index 30 to 31, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New cstate: current_term: 3 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:28.954962 25422 tablet_service.cc:1552] Processing DeleteTablet for tablet 22e5c53fdb9d496b8b87aa938e9a4c3f with delete_type TABLET_DATA_TOMBSTONED (TS 4af27de48bec4590bfcd8d71056814eb not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:28.955674 26291 tablet_replica.cc:333] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:28.955873 26291 raft_consensus.cc:2243] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:28.956120 26291 raft_consensus.cc:2272] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:28.956884 26291 ts_tablet_manager.cc:1916] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:28.961339 26291 ts_tablet_manager.cc:1929] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.30
I20251210 21:05:28.961441 26291 log.cc:1199] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 4af27de48bec4590bfcd8d71056814eb: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/22e5c53fdb9d496b8b87aa938e9a4c3f
I20251210 21:05:28.962034 25247 catalog_manager.cc:4985] TS 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): tablet 22e5c53fdb9d496b8b87aa938e9a4c3f (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:29.227631 25750 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 LEARNER]: Advancing to term 4
I20251210 21:05:29.229837 25750 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251210 21:05:29.230551 26224 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:29.232995 26224 raft_consensus.cc:1064] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
W20251210 21:05:29.233114 26224 raft_consensus.cc:1068] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: Unable to promote non-voter dbc0c6bd96c441e28d5a340d85fbe323: Illegal state: Leader has not yet committed an operation in its own term
I20251210 21:05:29.296181 25509 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251210 21:05:29.296706 26224 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:29.300628 25437 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251210 21:05:29.301280 26226 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:29.303617 26224 raft_consensus.cc:1064] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
I20251210 21:05:29.304374 26224 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 4.30, Last appended by leader: 29, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:29.306246 25509 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:29.306385 25437 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:29.306386 25750 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:29.306893 26298 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:29.307083 26224 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:29.307479 26293 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:29.309996 26224 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Committing config change with OpId 4.31: config changed from index 28 to 31, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.313470 25435 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.313632 25750 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.316110 25509 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.318306 25259 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 28 to 31, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:29.322676 25363 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:29.324186 25750 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:29.324404 25435 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:29.324606 26298 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:29.324882 26226 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:29.326905 26293 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.327521 25750 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.330245 25249 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 4810894f161642aba143677e92893e72 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:29.330876 25258 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New cstate: current_term: 4 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.330936 25435 raft_consensus.cc:2955] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:29.334599 25499 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810894f161642aba143677e92893e72 with delete_type TABLET_DATA_TOMBSTONED (TS 56e19ce44c0c4f08b1eb440e1ef3e9cd not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:29.335254 26299 tablet_replica.cc:333] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:29.335418 26299 raft_consensus.cc:2243] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:29.335805 26299 raft_consensus.cc:2272] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:29.336655 26299 ts_tablet_manager.cc:1916] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:29.341028 26299 ts_tablet_manager.cc:1929] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.31
I20251210 21:05:29.341145 26299 log.cc:1199] T 4810894f161642aba143677e92893e72 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/4810894f161642aba143677e92893e72
I20251210 21:05:29.341768 25247 catalog_manager.cc:4985] TS 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213): tablet 4810894f161642aba143677e92893e72 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:29.761868 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:29.765048 25363 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:29.765288 25363 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:29.766391 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:29.766552 25435 raft_consensus.cc:606] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Received request to transfer leadership to TS a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:29.766953 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251210 21:05:29.767206 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:29.784180 26233 raft_consensus.cc:993] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower a6abef32d32c46478bc3f2b9aa589efb to start an election
I20251210 21:05:29.784353 26233 raft_consensus.cc:1081] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Signalling peer a6abef32d32c46478bc3f2b9aa589efb to start an election
I20251210 21:05:29.784868 25825 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746"
dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
from {username='slave'} at 127.0.0.1:46918
I20251210 21:05:29.785040 25825 raft_consensus.cc:493] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:29.785138 25825 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:29.787180 25825 raft_consensus.cc:515] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:29.787770 25825 leader_election.cc:290] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [CANDIDATE]: Term 4 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213)
I20251210 21:05:29.788062 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "a6abef32d32c46478bc3f2b9aa589efb" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:29.788211 25435 raft_consensus.cc:3055] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Stepping down as leader of term 3
I20251210 21:05:29.788272 25435 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:29.788240 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7e5750d29e94bad943f054f32b40746" candidate_uuid: "a6abef32d32c46478bc3f2b9aa589efb" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:29.788384 25509 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:29.788497 25435 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:29.788784 25435 raft_consensus.cc:3060] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:29.790221 25509 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6abef32d32c46478bc3f2b9aa589efb in term 4.
I20251210 21:05:29.790560 25435 raft_consensus.cc:2468] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6abef32d32c46478bc3f2b9aa589efb in term 4.
I20251210 21:05:29.790577 25788 leader_election.cc:304] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e19ce44c0c4f08b1eb440e1ef3e9cd, a6abef32d32c46478bc3f2b9aa589efb; no voters:
I20251210 21:05:29.790824 26231 raft_consensus.cc:2804] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Leader election won for term 4
I20251210 21:05:29.791162 26231 raft_consensus.cc:697] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEADER]: Becoming Leader. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Running, Role: LEADER
I20251210 21:05:29.791396 26231 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:29.793824 25257 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb reported cstate change: term changed from 3 to 4, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69). New cstate: current_term: 4 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:29.805481 26224 raft_consensus.cc:993] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:29.805632 26224 raft_consensus.cc:1081] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:29.806200 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43858
I20251210 21:05:29.806366 25435 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:29.806450 25435 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:29.808367 25435 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:29.808934 25435 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 5 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845)
I20251210 21:05:29.809319 25750 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
I20251210 21:05:29.809345 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:29.809473 25750 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:29.809543 25363 raft_consensus.cc:3055] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Stepping down as leader of term 4
I20251210 21:05:29.809634 25363 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:29.809885 25363 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:29.810223 25363 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:29.811244 25750 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 5.
I20251210 21:05:29.811640 25406 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, dbc0c6bd96c441e28d5a340d85fbe323; no voters:
I20251210 21:05:29.811909 26233 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Leader election won for term 5
I20251210 21:05:29.812081 26233 raft_consensus.cc:697] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:29.812089 25363 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 5.
I20251210 21:05:29.812357 26233 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:29.814321 25257 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 4 to 5, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 5 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:29.955699 25750 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:29.958351 25898 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20251210 21:05:29.958828 25435 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20251210 21:05:29.958906 26304 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251210 21:05:29.959743 26305 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251210 21:05:29.962224 26304 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Committing config change with OpId 4.36: config changed from index 35 to 36, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:29.962865 25435 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.36: config changed from index 35 to 36, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:29.963474 25898 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.36: config changed from index 35 to 36, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
W20251210 21:05:29.965898 25713 consensus_peers.cc:597] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f1698d0d3bd4765bc0897771cb0ceee. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:29.968255 25257 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 35 to 36, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New cstate: current_term: 4 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:29.975544 25898 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:29.977788 25363 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251210 21:05:29.977807 25509 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
W20251210 21:05:29.977993 25865 consensus_peers.cc:597] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 -> Peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845): Couldn't send request to peer dbc0c6bd96c441e28d5a340d85fbe323. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6e20042b566343a7acdbe06f4dcb0c86. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:29.978354 26313 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:29.978606 26222 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:29.981108 26313 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:29.981774 25363 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:29.984472 25509 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } } }
I20251210 21:05:29.985445 25259 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 reported cstate change: config changed from index 29 to 31, NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:29.987200 25363 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 3.28, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:29.991180 25435 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251210 21:05:29.991497 25509 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
W20251210 21:05:29.991562 25328 consensus_peers.cc:597] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 -> Peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341): Couldn't send request to peer a6abef32d32c46478bc3f2b9aa589efb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b13da827577e49569e304f0cbfb5e9df. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:29.991858 26224 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:29.992137 26293 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251210 21:05:29.994864 26224 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:29.995172 25509 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:29.996327 25435 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
I20251210 21:05:29.999066 25257 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: config changed from index -1 to 29, NON_VOTER a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) added. New cstate: current_term: 3 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } } }
W20251210 21:05:30.001112 25311 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251210 21:05:30.006683 25311 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet e7e5750d29e94bad943f054f32b40746, TS 4af27de48bec4590bfcd8d71056814eb -> TS 90c04d3cdd4b4cb18b2a265365b1b094 move failed, destination replica disappeared from tablet's Raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
W20251210 21:05:30.006785 25311 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet e7e5750d29e94bad943f054f32b40746, TS 4af27de48bec4590bfcd8d71056814eb -> TS 90c04d3cdd4b4cb18b2a265365b1b094 move failed, destination replica disappeared from tablet's Raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:30.187230 25435 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:30.187798 26231 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:30.192857 25509 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251210 21:05:30.193514 26231 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:30.224166 25750 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 5 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251210 21:05:30.224733 26233 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:30.228863 25363 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251210 21:05:30.229605 26300 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:30.388172 26329 ts_tablet_manager.cc:933] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: Initiating tablet copy from peer dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845)
I20251210 21:05:30.388873 26329 tablet_copy_client.cc:323] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.68:37845
I20251210 21:05:30.394937 25760 tablet_copy_service.cc:140] P dbc0c6bd96c441e28d5a340d85fbe323: Received BeginTabletCopySession request for tablet 4f1698d0d3bd4765bc0897771cb0ceee from peer a6abef32d32c46478bc3f2b9aa589efb ({username='slave'} at 127.0.0.1:41376)
I20251210 21:05:30.395140 25760 tablet_copy_service.cc:161] P dbc0c6bd96c441e28d5a340d85fbe323: Beginning new tablet copy session on tablet 4f1698d0d3bd4765bc0897771cb0ceee from peer a6abef32d32c46478bc3f2b9aa589efb at {username='slave'} at 127.0.0.1:41376: session id = a6abef32d32c46478bc3f2b9aa589efb-4f1698d0d3bd4765bc0897771cb0ceee
I20251210 21:05:30.397754 25760 tablet_copy_source_session.cc:215] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:30.398815 26329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f1698d0d3bd4765bc0897771cb0ceee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:30.403793 26329 tablet_copy_client.cc:806] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:30.404049 26329 tablet_copy_client.cc:670] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:30.405546 26330 tablet_copy_client.cc:1025] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:30.491382 26333 ts_tablet_manager.cc:933] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: Initiating tablet copy from peer 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549)
I20251210 21:05:30.491966 26333 tablet_copy_client.cc:323] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.70:38549
I20251210 21:05:30.492604 25909 tablet_copy_service.cc:140] P 90c04d3cdd4b4cb18b2a265365b1b094: Received BeginTabletCopySession request for tablet 6e20042b566343a7acdbe06f4dcb0c86 from peer dbc0c6bd96c441e28d5a340d85fbe323 ({username='slave'} at 127.0.0.1:58738)
I20251210 21:05:30.492741 25909 tablet_copy_service.cc:161] P 90c04d3cdd4b4cb18b2a265365b1b094: Beginning new tablet copy session on tablet 6e20042b566343a7acdbe06f4dcb0c86 from peer dbc0c6bd96c441e28d5a340d85fbe323 at {username='slave'} at 127.0.0.1:58738: session id = dbc0c6bd96c441e28d5a340d85fbe323-6e20042b566343a7acdbe06f4dcb0c86
I20251210 21:05:30.495174 25909 tablet_copy_source_session.cc:215] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:30.496222 26333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20042b566343a7acdbe06f4dcb0c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:30.501087 26333 tablet_copy_client.cc:806] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:30.501366 26333 tablet_copy_client.cc:670] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:30.502890 26334 tablet_copy_client.cc:1025] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:30.561546 26335 ts_tablet_manager.cc:933] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: Initiating tablet copy from peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411)
I20251210 21:05:30.562211 26335 tablet_copy_client.cc:323] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:41411
I20251210 21:05:30.568153 25375 tablet_copy_service.cc:140] P 128e4ab89fe449e8b5fdc68590b7ac89: Received BeginTabletCopySession request for tablet b13da827577e49569e304f0cbfb5e9df from peer a6abef32d32c46478bc3f2b9aa589efb ({username='slave'} at 127.0.0.1:46576)
I20251210 21:05:30.568297 25375 tablet_copy_service.cc:161] P 128e4ab89fe449e8b5fdc68590b7ac89: Beginning new tablet copy session on tablet b13da827577e49569e304f0cbfb5e9df from peer a6abef32d32c46478bc3f2b9aa589efb at {username='slave'} at 127.0.0.1:46576: session id = a6abef32d32c46478bc3f2b9aa589efb-b13da827577e49569e304f0cbfb5e9df
I20251210 21:05:30.570518 25375 tablet_copy_source_session.cc:215] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:30.571741 26335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b13da827577e49569e304f0cbfb5e9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:30.577418 26335 tablet_copy_client.cc:806] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:30.577636 26335 tablet_copy_client.cc:670] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:30.578986 26336 tablet_copy_client.cc:1025] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:31.767509 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:31.770381 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:31.770591 25435 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:31.771019 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:31.771196 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:31.839727 26349 raft_consensus.cc:993] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:31.839898 26349 raft_consensus.cc:1081] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:31.840405 25364 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:31.840551 25364 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:31.840638 25364 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:31.842744 25364 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:31.843274 25364 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 6 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845)
I20251210 21:05:31.843691 25749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
I20251210 21:05:31.843714 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:31.843835 25749 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:31.843969 25435 raft_consensus.cc:3055] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 LEADER]: Stepping down as leader of term 5
I20251210 21:05:31.844039 25435 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:31.844269 25435 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:31.844589 25435 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:31.846379 25749 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 6.
I20251210 21:05:31.846796 25331 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, dbc0c6bd96c441e28d5a340d85fbe323; no voters:
I20251210 21:05:31.846843 25435 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 6.
I20251210 21:05:31.847091 26348 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Leader election won for term 6
I20251210 21:05:31.847222 26348 raft_consensus.cc:697] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:31.847477 26348 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:31.850096 25258 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 5 to 6, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 6 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:32.281860 25435 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251210 21:05:32.282298 26348 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:32.287947 25749 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 6 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251210 21:05:32.288574 26348 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:32.405963 26329 tablet_copy_client.cc:538] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:32.408639 26329 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap starting.
I20251210 21:05:32.444633 26329 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=89 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:32.445042 26329 tablet_bootstrap.cc:492] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap complete.
I20251210 21:05:32.445302 26329 ts_tablet_manager.cc:1403] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: Time spent bootstrapping tablet: real 0.037s user 0.035s sys 0.000s
I20251210 21:05:32.446020 26329 raft_consensus.cc:359] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:32.446204 26329 raft_consensus.cc:740] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Initialized, Role: LEARNER
I20251210 21:05:32.446374 26329 consensus_queue.cc:260] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 4.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:32.446897 26329 ts_tablet_manager.cc:1434] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:05:32.447660 25760 tablet_copy_service.cc:342] P dbc0c6bd96c441e28d5a340d85fbe323: Request end of tablet copy session a6abef32d32c46478bc3f2b9aa589efb-4f1698d0d3bd4765bc0897771cb0ceee received from {username='slave'} at 127.0.0.1:41376
I20251210 21:05:32.447769 25760 tablet_copy_service.cc:434] P dbc0c6bd96c441e28d5a340d85fbe323: ending tablet copy session a6abef32d32c46478bc3f2b9aa589efb-4f1698d0d3bd4765bc0897771cb0ceee on tablet 4f1698d0d3bd4765bc0897771cb0ceee with peer a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:32.503237 26333 tablet_copy_client.cc:538] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:32.506310 26333 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap starting.
I20251210 21:05:32.542906 26333 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:32.543313 26333 tablet_bootstrap.cc:492] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: Bootstrap complete.
I20251210 21:05:32.543608 26333 ts_tablet_manager.cc:1403] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent bootstrapping tablet: real 0.037s user 0.038s sys 0.001s
I20251210 21:05:32.544420 26333 raft_consensus.cc:359] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:32.544631 26333 raft_consensus.cc:740] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbc0c6bd96c441e28d5a340d85fbe323, State: Initialized, Role: LEARNER
I20251210 21:05:32.544816 26333 consensus_queue.cc:260] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: true } }
I20251210 21:05:32.545300 26333 ts_tablet_manager.cc:1434] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:05:32.545912 25909 tablet_copy_service.cc:342] P 90c04d3cdd4b4cb18b2a265365b1b094: Request end of tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-6e20042b566343a7acdbe06f4dcb0c86 received from {username='slave'} at 127.0.0.1:58738
I20251210 21:05:32.546065 25909 tablet_copy_service.cc:434] P 90c04d3cdd4b4cb18b2a265365b1b094: ending tablet copy session dbc0c6bd96c441e28d5a340d85fbe323-6e20042b566343a7acdbe06f4dcb0c86 on tablet 6e20042b566343a7acdbe06f4dcb0c86 with peer dbc0c6bd96c441e28d5a340d85fbe323
I20251210 21:05:32.579352 26335 tablet_copy_client.cc:538] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:32.582350 26335 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap starting.
I20251210 21:05:32.613003 26335 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=109 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:32.613444 26335 tablet_bootstrap.cc:492] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: Bootstrap complete.
I20251210 21:05:32.613703 26335 ts_tablet_manager.cc:1403] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: Time spent bootstrapping tablet: real 0.031s user 0.027s sys 0.005s
I20251210 21:05:32.614403 26335 raft_consensus.cc:359] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:32.614593 26335 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a6abef32d32c46478bc3f2b9aa589efb, State: Initialized, Role: LEARNER
I20251210 21:05:32.614825 26335 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: NON_VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: true } }
I20251210 21:05:32.615315 26335 ts_tablet_manager.cc:1434] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:05:32.615911 25375 tablet_copy_service.cc:342] P 128e4ab89fe449e8b5fdc68590b7ac89: Request end of tablet copy session a6abef32d32c46478bc3f2b9aa589efb-b13da827577e49569e304f0cbfb5e9df received from {username='slave'} at 127.0.0.1:46576
I20251210 21:05:32.616066 25375 tablet_copy_service.cc:434] P 128e4ab89fe449e8b5fdc68590b7ac89: ending tablet copy session a6abef32d32c46478bc3f2b9aa589efb-b13da827577e49569e304f0cbfb5e9df on tablet b13da827577e49569e304f0cbfb5e9df with peer a6abef32d32c46478bc3f2b9aa589efb
I20251210 21:05:32.779695 25825 raft_consensus.cc:1217] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEARNER]: Deduplicated request from leader. Original: 4.35->[4.36-4.36] Dedup: 4.36->[]
I20251210 21:05:32.834745 25825 raft_consensus.cc:1217] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Deduplicated request from leader. Original: 3.28->[3.29-3.29] Dedup: 3.29->[]
I20251210 21:05:32.923352 25749 raft_consensus.cc:1217] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.30->[2.31-2.31] Dedup: 2.31->[]
I20251210 21:05:33.367612 26366 raft_consensus.cc:1064] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: attempting to promote NON_VOTER a6abef32d32c46478bc3f2b9aa589efb to VOTER
I20251210 21:05:33.368352 26366 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 27, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:33.371073 25435 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:33.371122 25507 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:33.371073 25825 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEARNER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251210 21:05:33.371833 26364 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:33.372385 26364 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:33.372732 26368 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251210 21:05:33.376345 26342 raft_consensus.cc:1064] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: attempting to promote NON_VOTER a6abef32d32c46478bc3f2b9aa589efb to VOTER
I20251210 21:05:33.376725 26368 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.377151 26342 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 4.36, Last appended by leader: 31, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:33.377210 25435 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.378913 25825 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.381836 25259 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.382793 25507 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } attrs { replace: true } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.390501 25363 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:33.390935 25825 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEARNER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 4 index: 37. (index mismatch)
I20251210 21:05:33.391091 25898 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 4 index: 37. (index mismatch)
I20251210 21:05:33.391537 26369 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251210 21:05:33.392063 26369 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251210 21:05:33.392305 25825 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:33.392361 25435 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:33.392818 26365 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:33.393255 26365 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:33.396415 25435 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 4 index: 37. (index mismatch)
I20251210 21:05:33.396288 26366 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.397743 26369 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251210 21:05:33.397789 25824 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.399230 25249 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet b13da827577e49569e304f0cbfb5e9df with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:33.399571 25437 raft_consensus.cc:2955] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.403591 25256 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: config changed from index 30 to 31, VOTER 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) evicted. New cstate: current_term: 3 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:33.407107 25499 tablet_service.cc:1552] Processing DeleteTablet for tablet b13da827577e49569e304f0cbfb5e9df with delete_type TABLET_DATA_TOMBSTONED (TS 56e19ce44c0c4f08b1eb440e1ef3e9cd not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:47088
I20251210 21:05:33.407755 26382 tablet_replica.cc:333] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:33.408033 26382 raft_consensus.cc:2243] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:33.408352 26382 raft_consensus.cc:2272] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:33.410385 26382 ts_tablet_manager.cc:1916] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:33.416203 26378 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Committing config change with OpId 4.37: config changed from index 36 to 37, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.417140 26382 ts_tablet_manager.cc:1929] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.30
I20251210 21:05:33.417250 26382 log.cc:1199] T b13da827577e49569e304f0cbfb5e9df P 56e19ce44c0c4f08b1eb440e1ef3e9cd: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/b13da827577e49569e304f0cbfb5e9df
I20251210 21:05:33.416946 25825 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Committing config change with OpId 4.37: config changed from index 36 to 37, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.417936 25247 catalog_manager.cc:4985] TS 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213): tablet b13da827577e49569e304f0cbfb5e9df (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:33.420543 25899 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.37: config changed from index 36 to 37, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.421006 25256 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 36 to 37, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:33.425000 25435 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.37: config changed from index 36 to 37, a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.427285 25750 consensus_queue.cc:237] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 4.37, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:33.428966 25825 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 37. Preceding OpId from leader: term: 4 index: 38. (index mismatch)
I20251210 21:05:33.429636 26342 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251210 21:05:33.429997 25899 raft_consensus.cc:1275] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Refusing update from remote peer dbc0c6bd96c441e28d5a340d85fbe323: Log matching property violated. Preceding OpId in replica: term: 4 index: 37. Preceding OpId from leader: term: 4 index: 38. (index mismatch)
I20251210 21:05:33.430418 26342 consensus_queue.cc:1048] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251210 21:05:33.432442 26369 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Committing config change with OpId 4.38: config changed from index 37 to 38, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.432948 25898 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.38: config changed from index 37 to 38, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.433286 25825 raft_consensus.cc:2955] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Committing config change with OpId 4.38: config changed from index 37 to 38, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } }
I20251210 21:05:33.435593 25310 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 4f1698d0d3bd4765bc0897771cb0ceee with cas_config_opid_index 37: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:33.436897 25256 catalog_manager.cc:5654] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 reported cstate change: config changed from index 37 to 38, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New cstate: current_term: 4 leader_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" committed_config { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:33.440644 25422 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f1698d0d3bd4765bc0897771cb0ceee with delete_type TABLET_DATA_TOMBSTONED (TS 4af27de48bec4590bfcd8d71056814eb not found in new config with opid_index 38) from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:33.441377 26383 tablet_replica.cc:333] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:33.441660 26383 raft_consensus.cc:2243] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:33.442056 26383 raft_consensus.cc:2272] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:33.442797 26383 ts_tablet_manager.cc:1916] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:33.447602 26383 ts_tablet_manager.cc:1929] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.37
I20251210 21:05:33.447705 26383 log.cc:1199] T 4f1698d0d3bd4765bc0897771cb0ceee P 4af27de48bec4590bfcd8d71056814eb: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/4f1698d0d3bd4765bc0897771cb0ceee
I20251210 21:05:33.448318 25247 catalog_manager.cc:4985] TS 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): tablet 4f1698d0d3bd4765bc0897771cb0ceee (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:33.478250 26384 raft_consensus.cc:1064] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: attempting to promote NON_VOTER dbc0c6bd96c441e28d5a340d85fbe323 to VOTER
I20251210 21:05:33.479044 26384 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:33.481138 25749 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 LEARNER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251210 21:05:33.481220 25507 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251210 21:05:33.481596 26384 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:33.481858 25363 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251210 21:05:33.482190 26343 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:33.482376 26385 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:33.485621 26385 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:33.489287 25749 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:33.489711 25258 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 reported cstate change: config changed from index 31 to 32, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:33.491922 25363 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:33.500427 25507 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:33.502591 25899 consensus_queue.cc:237] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:33.504336 25749 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251210 21:05:33.504424 25507 raft_consensus.cc:1275] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Refusing update from remote peer 90c04d3cdd4b4cb18b2a265365b1b094: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251210 21:05:33.505182 26384 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:33.505605 26390 consensus_queue.cc:1048] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:33.507702 26385 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:33.508160 25507 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:33.508778 25749 raft_consensus.cc:2955] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } } }
I20251210 21:05:33.510484 25310 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 6e20042b566343a7acdbe06f4dcb0c86 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:33.511790 25257 catalog_manager.cc:5654] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 reported cstate change: config changed from index 32 to 33, VOTER 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) evicted. New cstate: current_term: 2 leader_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:33.514907 25352 tablet_service.cc:1552] Processing DeleteTablet for tablet 6e20042b566343a7acdbe06f4dcb0c86 with delete_type TABLET_DATA_TOMBSTONED (TS 128e4ab89fe449e8b5fdc68590b7ac89 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:39012
I20251210 21:05:33.515625 26391 tablet_replica.cc:333] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:33.515787 26391 raft_consensus.cc:2243] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:33.515965 26391 raft_consensus.cc:2272] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:33.516601 26391 ts_tablet_manager.cc:1916] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:33.520596 26391 ts_tablet_manager.cc:1929] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.32
I20251210 21:05:33.520679 26391 log.cc:1199] T 6e20042b566343a7acdbe06f4dcb0c86 P 128e4ab89fe449e8b5fdc68590b7ac89: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/6e20042b566343a7acdbe06f4dcb0c86
I20251210 21:05:33.521253 25249 catalog_manager.cc:4985] TS 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): tablet 6e20042b566343a7acdbe06f4dcb0c86 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:33.771612 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:33.774326 25363 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:33.774554 25363 raft_consensus.cc:606] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:33.775002 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:33.775257 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:33.981556 26366 raft_consensus.cc:993] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:33.981686 26366 raft_consensus.cc:1081] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:33.982432 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43858
I20251210 21:05:33.982652 25435 raft_consensus.cc:493] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:33.982791 25435 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:33.984870 25435 raft_consensus.cc:515] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:33.985563 25435 leader_election.cc:290] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 4 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:33.985948 25825 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
I20251210 21:05:33.985908 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:33.986119 25825 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:33.986148 25363 raft_consensus.cc:3055] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Stepping down as leader of term 3
I20251210 21:05:33.986224 25363 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:33.986474 25363 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:33.986807 25363 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:33.988910 25825 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 4.
I20251210 21:05:33.989236 25363 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 4.
I20251210 21:05:33.989344 25403 leader_election.cc:304] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, a6abef32d32c46478bc3f2b9aa589efb; no voters:
I20251210 21:05:33.989619 26349 raft_consensus.cc:2804] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Leader election won for term 4
I20251210 21:05:33.990072 26349 raft_consensus.cc:697] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:33.990309 26349 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:33.992290 25258 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 3 to 4, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 4 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:34.518188 25825 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
W20251210 21:05:34.520365 25791 consensus_peers.cc:597] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb -> Peer 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70:38549): Couldn't send request to peer 90c04d3cdd4b4cb18b2a265365b1b094. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7e5750d29e94bad943f054f32b40746. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:34.520357 25507 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:34.520500 25435 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:34.520925 26396 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:34.521199 26395 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:34.523725 26395 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index 30 to 32, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:34.524438 25507 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 30 to 32, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:34.524516 25435 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 30 to 32, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:34.529013 25256 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: config changed from index 30 to 32, NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) added. New cstate: current_term: 4 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } } }
I20251210 21:05:34.553766 25363 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:34.554302 26349 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:34.559080 25825 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:34.568053 26405 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:34.988111 26412 ts_tablet_manager.cc:933] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: Initiating tablet copy from peer a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:34.989063 26412 tablet_copy_client.cc:323] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.69:45341
I20251210 21:05:34.989715 25835 tablet_copy_service.cc:140] P a6abef32d32c46478bc3f2b9aa589efb: Received BeginTabletCopySession request for tablet e7e5750d29e94bad943f054f32b40746 from peer 90c04d3cdd4b4cb18b2a265365b1b094 ({username='slave'} at 127.0.0.1:56498)
I20251210 21:05:34.989856 25835 tablet_copy_service.cc:161] P a6abef32d32c46478bc3f2b9aa589efb: Beginning new tablet copy session on tablet e7e5750d29e94bad943f054f32b40746 from peer 90c04d3cdd4b4cb18b2a265365b1b094 at {username='slave'} at 127.0.0.1:56498: session id = 90c04d3cdd4b4cb18b2a265365b1b094-e7e5750d29e94bad943f054f32b40746
I20251210 21:05:34.991765 25835 tablet_copy_source_session.cc:215] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:34.992839 26412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e5750d29e94bad943f054f32b40746. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:34.997983 26412 tablet_copy_client.cc:806] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:34.998250 26412 tablet_copy_client.cc:670] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:34.999809 26413 tablet_copy_client.cc:1025] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Injecting latency into file download: 2000
I20251210 21:05:35.775632 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:35.779213 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:35.779415 25435 raft_consensus.cc:606] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:35.779856 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:35.780097 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:35.949841 26409 raft_consensus.cc:993] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:35.950014 26409 raft_consensus.cc:1081] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:35.950700 25363 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39030
I20251210 21:05:35.950907 25363 raft_consensus.cc:493] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:35.951119 25363 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:35.953733 25363 raft_consensus.cc:515] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:35.954360 25363 leader_election.cc:290] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 5 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:35.954618 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:35.954785 25435 raft_consensus.cc:3055] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Stepping down as leader of term 4
I20251210 21:05:35.954762 25824 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
I20251210 21:05:35.954942 25435 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:35.954993 25824 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:35.955173 25435 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:35.955503 25435 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Advancing to term 5
I20251210 21:05:35.957612 25435 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 5.
I20251210 21:05:35.957607 25824 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 5.
I20251210 21:05:35.958024 25328 leader_election.cc:304] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, 4af27de48bec4590bfcd8d71056814eb; no voters:
I20251210 21:05:35.958317 26366 raft_consensus.cc:2804] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Leader election won for term 5
I20251210 21:05:35.958663 26366 raft_consensus.cc:697] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:35.958894 26366 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:35.961464 25257 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 4 to 5, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 5 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:36.357671 25435 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251210 21:05:36.358276 26366 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:36.368839 25824 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 5 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251210 21:05:36.369323 26366 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:37.000155 26412 tablet_copy_client.cc:538] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:37.003646 26412 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap starting.
I20251210 21:05:37.044211 26412 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=101 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:37.044612 26412 tablet_bootstrap.cc:492] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: Bootstrap complete.
I20251210 21:05:37.044945 26412 ts_tablet_manager.cc:1403] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent bootstrapping tablet: real 0.041s user 0.035s sys 0.004s
I20251210 21:05:37.045799 26412 raft_consensus.cc:359] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:37.046005 26412 raft_consensus.cc:740] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 90c04d3cdd4b4cb18b2a265365b1b094, State: Initialized, Role: LEARNER
I20251210 21:05:37.046172 26412 consensus_queue.cc:260] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: NON_VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: true } }
I20251210 21:05:37.046619 26412 ts_tablet_manager.cc:1434] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:05:37.047369 25835 tablet_copy_service.cc:342] P a6abef32d32c46478bc3f2b9aa589efb: Request end of tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-e7e5750d29e94bad943f054f32b40746 received from {username='slave'} at 127.0.0.1:56498
I20251210 21:05:37.047511 25835 tablet_copy_service.cc:434] P a6abef32d32c46478bc3f2b9aa589efb: ending tablet copy session 90c04d3cdd4b4cb18b2a265365b1b094-e7e5750d29e94bad943f054f32b40746 on tablet e7e5750d29e94bad943f054f32b40746 with peer 90c04d3cdd4b4cb18b2a265365b1b094
I20251210 21:05:37.518761 25898 raft_consensus.cc:1217] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.31->[4.32-4.32] Dedup: 4.32->[]
I20251210 21:05:37.780441 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:37.783610 25363 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:37.783888 25363 raft_consensus.cc:606] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:37.784318 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:37.784535 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:37.817991 26434 raft_consensus.cc:993] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:37.818143 26434 raft_consensus.cc:1081] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:37.818730 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43858
I20251210 21:05:37.818950 25435 raft_consensus.cc:493] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:37.819054 25435 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:37.821409 25435 raft_consensus.cc:515] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:37.822028 25435 leader_election.cc:290] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 6 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:37.822366 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:37.822396 25825 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b13da827577e49569e304f0cbfb5e9df" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
I20251210 21:05:37.822542 25363 raft_consensus.cc:3055] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Stepping down as leader of term 5
I20251210 21:05:37.822556 25825 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:37.822614 25363 raft_consensus.cc:740] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:37.822830 25363 consensus_queue.cc:260] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:37.823169 25363 raft_consensus.cc:3060] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 5 FOLLOWER]: Advancing to term 6
I20251210 21:05:37.825404 25825 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 6.
I20251210 21:05:37.825799 25363 raft_consensus.cc:2468] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 6.
I20251210 21:05:37.825826 25403 leader_election.cc:304] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, a6abef32d32c46478bc3f2b9aa589efb; no voters:
I20251210 21:05:37.826267 26409 raft_consensus.cc:2804] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Leader election won for term 6
I20251210 21:05:37.826633 26409 raft_consensus.cc:697] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:37.826869 26409 consensus_queue.cc:237] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:37.829217 25257 catalog_manager.cc:5654] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 5 to 6, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 6 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:38.133569 26395 raft_consensus.cc:1064] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: attempting to promote NON_VOTER 90c04d3cdd4b4cb18b2a265365b1b094 to VOTER
I20251210 21:05:38.134308 26395 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 30, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:38.137068 25898 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 LEARNER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251210 21:05:38.137701 25435 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251210 21:05:38.137753 25507 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251210 21:05:38.138001 26442 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:38.138320 26395 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:38.138602 26441 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251210 21:05:38.142490 26441 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:38.143352 25507 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:38.146394 25435 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:38.147544 25898 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:38.148258 25257 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb reported cstate change: config changed from index 32 to 33, 90c04d3cdd4b4cb18b2a265365b1b094 (127.21.127.70) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:38.152645 25824 consensus_queue.cc:237] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } }
I20251210 21:05:38.154605 25898 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251210 21:05:38.155146 26441 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:38.155555 25507 raft_consensus.cc:1275] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer a6abef32d32c46478bc3f2b9aa589efb: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251210 21:05:38.156003 26444 consensus_queue.cc:1048] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:38.157302 26444 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:38.157802 25899 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:38.160533 25305 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet e7e5750d29e94bad943f054f32b40746 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:38.161902 25256 catalog_manager.cc:5654] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb reported cstate change: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New cstate: current_term: 4 leader_uuid: "a6abef32d32c46478bc3f2b9aa589efb" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:05:38.162129 25507 raft_consensus.cc:2955] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } } peers { permanent_uuid: "90c04d3cdd4b4cb18b2a265365b1b094" member_type: VOTER last_known_addr { host: "127.21.127.70" port: 38549 } attrs { promote: false } } }
I20251210 21:05:38.165458 25422 tablet_service.cc:1552] Processing DeleteTablet for tablet e7e5750d29e94bad943f054f32b40746 with delete_type TABLET_DATA_TOMBSTONED (TS 4af27de48bec4590bfcd8d71056814eb not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:43826
I20251210 21:05:38.166164 26448 tablet_replica.cc:333] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:38.166350 26448 raft_consensus.cc:2243] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:38.166590 26448 raft_consensus.cc:2272] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:38.168028 26448 ts_tablet_manager.cc:1916] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:38.172260 26448 ts_tablet_manager.cc:1929] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.33
I20251210 21:05:38.172369 26448 log.cc:1199] T e7e5750d29e94bad943f054f32b40746 P 4af27de48bec4590bfcd8d71056814eb: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/e7e5750d29e94bad943f054f32b40746
I20251210 21:05:38.173051 25247 catalog_manager.cc:4985] TS 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981): tablet e7e5750d29e94bad943f054f32b40746 (table test-workload [id=9cb6ac477b914209a88da6ddd266da43]) successfully deleted
I20251210 21:05:38.220932 25825 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 6 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251210 21:05:38.221498 26409 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:38.226099 25363 raft_consensus.cc:1275] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251210 21:05:38.226934 26451 consensus_queue.cc:1048] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251210 21:05:39.785050 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:39.787846 25435 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
mode: GRACEFUL
new_leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:33328
I20251210 21:05:39.788057 25435 raft_consensus.cc:606] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Received request to transfer leadership to TS 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:39.788621 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:39.788837 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:40.206749 26451 raft_consensus.cc:993] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: : Instructing follower 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:40.206980 26451 raft_consensus.cc:1081] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Signalling peer 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:40.207659 25507 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:47096
I20251210 21:05:40.207896 25507 raft_consensus.cc:493] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:40.207986 25507 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:40.209833 25507 raft_consensus.cc:515] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:40.210474 25507 leader_election.cc:290] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Requested vote from peers 4af27de48bec4590bfcd8d71056814eb (127.21.127.66:39981), dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845)
I20251210 21:05:40.210958 25749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
I20251210 21:05:40.210975 25435 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b65abf7cc0484fcda919a741e23928a7" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
I20251210 21:05:40.211153 25749 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:40.211187 25435 raft_consensus.cc:3055] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:05:40.211254 25435 raft_consensus.cc:740] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:40.211509 25435 consensus_queue.cc:260] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:40.211788 25435 raft_consensus.cc:3060] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:05:40.213416 25749 raft_consensus.cc:2468] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:40.213708 25435 raft_consensus.cc:2468] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 3.
I20251210 21:05:40.213790 25480 leader_election.cc:304] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e19ce44c0c4f08b1eb440e1ef3e9cd, dbc0c6bd96c441e28d5a340d85fbe323; no voters:
I20251210 21:05:40.214025 26417 raft_consensus.cc:2804] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:05:40.214411 26417 raft_consensus.cc:697] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:40.214653 26417 consensus_queue.cc:237] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:40.217000 25258 catalog_manager.cc:5654] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 2 to 3, leader changed from 4af27de48bec4590bfcd8d71056814eb (127.21.127.66) to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 3 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:40.612000 25435 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:40.612519 26417 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:40.616190 25749 raft_consensus.cc:1275] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251210 21:05:40.616688 26417 consensus_queue.cc:1048] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251210 21:05:41.789259 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:41.792042 25507 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:41.792256 25507 raft_consensus.cc:606] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:41.792780 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:41.792989 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:41.958353 26458 raft_consensus.cc:993] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:41.958506 26458 raft_consensus.cc:1081] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:41.959170 25363 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39018
I20251210 21:05:41.959365 25363 raft_consensus.cc:493] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:41.959494 25363 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:41.961354 25363 raft_consensus.cc:515] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:41.961994 25363 leader_election.cc:290] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 4 election: Requested vote from peers 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213), a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:41.962450 25507 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:41.962529 25825 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22e5c53fdb9d496b8b87aa938e9a4c3f" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
I20251210 21:05:41.962608 25507 raft_consensus.cc:3055] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Stepping down as leader of term 3
I20251210 21:05:41.962658 25507 raft_consensus.cc:740] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:41.962692 25825 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:41.962854 25507 consensus_queue.cc:260] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:41.963193 25507 raft_consensus.cc:3060] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Advancing to term 4
I20251210 21:05:41.964710 25825 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 4.
I20251210 21:05:41.965032 25507 raft_consensus.cc:2468] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 4.
I20251210 21:05:41.965103 25328 leader_election.cc:304] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, a6abef32d32c46478bc3f2b9aa589efb; no voters:
I20251210 21:05:41.965340 26455 raft_consensus.cc:2804] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Leader election won for term 4
I20251210 21:05:41.965705 26455 raft_consensus.cc:697] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:41.966029 26455 consensus_queue.cc:237] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:41.968341 25258 catalog_manager.cc:5654] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 3 to 4, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 4 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:42.435937 25507 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:42.436434 26455 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:42.441365 25825 raft_consensus.cc:1275] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251210 21:05:42.441859 26455 consensus_queue.cc:1048] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251210 21:05:43.793401 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:43.796320 25363 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:43.796522 25363 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 LEADER]: Received request to transfer leadership to TS 56e19ce44c0c4f08b1eb440e1ef3e9cd
I20251210 21:05:43.797031 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:43.797271 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:43.862799 26483 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:43.862941 26483 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 LEADER]: Signalling peer 56e19ce44c0c4f08b1eb440e1ef3e9cd to start an election
I20251210 21:05:43.863641 25509 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
from {username='slave'} at 127.0.0.1:47102
I20251210 21:05:43.863826 25509 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:43.863929 25509 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 7 FOLLOWER]: Advancing to term 8
I20251210 21:05:43.866003 25509 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:43.866703 25509 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 8 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:43.867089 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 34 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:43.867216 25825 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 34 } } ignore_live_leader: true dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
I20251210 21:05:43.867259 25363 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 LEADER]: Stepping down as leader of term 7
I20251210 21:05:43.867373 25363 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:43.867383 25825 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 7 FOLLOWER]: Advancing to term 8
I20251210 21:05:43.867643 25363 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 7.34, Last appended by leader: 34, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:43.867959 25363 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 FOLLOWER]: Advancing to term 8
I20251210 21:05:43.869553 25825 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 8.
I20251210 21:05:43.869829 25363 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e19ce44c0c4f08b1eb440e1ef3e9cd in term 8.
I20251210 21:05:43.870002 25477 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e19ce44c0c4f08b1eb440e1ef3e9cd, a6abef32d32c46478bc3f2b9aa589efb; no voters:
I20251210 21:05:43.870307 26458 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 FOLLOWER]: Leader election won for term 8
I20251210 21:05:43.870669 26458 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 LEADER]: Becoming Leader. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:43.870983 26458 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 7.34, Last appended by leader: 34, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:43.873632 25256 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd reported cstate change: term changed from 7 to 8, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67). New cstate: current_term: 8 leader_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:44.496346 25825 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 8 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 34. Preceding OpId from leader: term: 8 index: 35. (index mismatch)
I20251210 21:05:44.496367 25363 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 8 FOLLOWER]: Refusing update from remote peer 56e19ce44c0c4f08b1eb440e1ef3e9cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 34. Preceding OpId from leader: term: 8 index: 35. (index mismatch)
I20251210 21:05:44.496985 26491 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251210 21:05:44.497332 26458 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251210 21:05:45.797701 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:45.800689 25509 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
mode: GRACEFUL
new_leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:47106
I20251210 21:05:45.800952 25509 raft_consensus.cc:606] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 LEADER]: Received request to transfer leadership to TS 128e4ab89fe449e8b5fdc68590b7ac89
I20251210 21:05:45.801445 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:45.801678 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:46.025228 26458 raft_consensus.cc:993] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: : Instructing follower 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:46.025444 26458 raft_consensus.cc:1081] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 LEADER]: Signalling peer 128e4ab89fe449e8b5fdc68590b7ac89 to start an election
I20251210 21:05:46.026158 25363 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
from {username='slave'} at 127.0.0.1:39018
I20251210 21:05:46.026392 25363 raft_consensus.cc:493] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:46.026535 25363 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 8 FOLLOWER]: Advancing to term 9
I20251210 21:05:46.028784 25363 raft_consensus.cc:515] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:46.029488 25363 leader_election.cc:290] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 9 election: Requested vote from peers 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67:36213), a6abef32d32c46478bc3f2b9aa589efb (127.21.127.69:45341)
I20251210 21:05:46.029953 25825 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 9 candidate_status { last_received { term: 8 index: 35 } } ignore_live_leader: true dest_uuid: "a6abef32d32c46478bc3f2b9aa589efb"
I20251210 21:05:46.030041 25509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "544270c4ded24ee2ae32f6aa4ec505c2" candidate_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" candidate_term: 9 candidate_status { last_received { term: 8 index: 35 } } ignore_live_leader: true dest_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd"
I20251210 21:05:46.030146 25825 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 8 FOLLOWER]: Advancing to term 9
I20251210 21:05:46.030182 25509 raft_consensus.cc:3055] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 LEADER]: Stepping down as leader of term 8
I20251210 21:05:46.030282 25509 raft_consensus.cc:740] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 56e19ce44c0c4f08b1eb440e1ef3e9cd, State: Running, Role: LEADER
I20251210 21:05:46.030542 25509 consensus_queue.cc:260] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 8.35, Last appended by leader: 35, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:46.030884 25509 raft_consensus.cc:3060] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 8 FOLLOWER]: Advancing to term 9
I20251210 21:05:46.032941 25825 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 9.
I20251210 21:05:46.033381 25328 leader_election.cc:304] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128e4ab89fe449e8b5fdc68590b7ac89, a6abef32d32c46478bc3f2b9aa589efb; no voters:
I20251210 21:05:46.033456 25509 raft_consensus.cc:2468] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 128e4ab89fe449e8b5fdc68590b7ac89 in term 9.
I20251210 21:05:46.033670 26483 raft_consensus.cc:2804] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 9 FOLLOWER]: Leader election won for term 9
I20251210 21:05:46.034060 26483 raft_consensus.cc:697] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 9 LEADER]: Becoming Leader. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:46.034343 26483 consensus_queue.cc:237] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 8.35, Last appended by leader: 35, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } }
I20251210 21:05:46.036664 25258 catalog_manager.cc:5654] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 reported cstate change: term changed from 8 to 9, leader changed from 56e19ce44c0c4f08b1eb440e1ef3e9cd (127.21.127.67) to 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65). New cstate: current_term: 9 leader_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:46.478189 25825 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 9 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 8 index: 35. Preceding OpId from leader: term: 9 index: 36. (index mismatch)
I20251210 21:05:46.478803 26483 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6abef32d32c46478bc3f2b9aa589efb" member_type: VOTER last_known_addr { host: "127.21.127.69" port: 45341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251210 21:05:46.483528 25509 raft_consensus.cc:1275] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 9 FOLLOWER]: Refusing update from remote peer 128e4ab89fe449e8b5fdc68590b7ac89: Log matching property violated. Preceding OpId in replica: term: 8 index: 35. Preceding OpId from leader: term: 9 index: 36. (index mismatch)
I20251210 21:05:46.484154 26501 consensus_queue.cc:1048] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e19ce44c0c4f08b1eb440e1ef3e9cd" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251210 21:05:47.802186 25316 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:47.805080 25363 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
mode: GRACEFUL
new_leader_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:55586
I20251210 21:05:47.805308 25363 raft_consensus.cc:606] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 LEADER]: Received request to transfer leadership to TS 4af27de48bec4590bfcd8d71056814eb
I20251210 21:05:47.805840 25316 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:47.806033 25316 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:47.829921 26501 raft_consensus.cc:993] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: : Instructing follower 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:47.830094 26501 raft_consensus.cc:1081] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 LEADER]: Signalling peer 4af27de48bec4590bfcd8d71056814eb to start an election
I20251210 21:05:47.830724 25435 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4810894f161642aba143677e92893e72"
dest_uuid: "4af27de48bec4590bfcd8d71056814eb"
from {username='slave'} at 127.0.0.1:43858
I20251210 21:05:47.830937 25435 raft_consensus.cc:493] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:47.831058 25435 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Advancing to term 7
I20251210 21:05:47.833012 25435 raft_consensus.cc:515] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:47.833801 25435 leader_election.cc:290] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 7 election: Requested vote from peers 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411), dbc0c6bd96c441e28d5a340d85fbe323 (127.21.127.68:37845)
I20251210 21:05:47.834244 25363 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 34 } } ignore_live_leader: true dest_uuid: "128e4ab89fe449e8b5fdc68590b7ac89"
I20251210 21:05:47.834381 25749 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4810894f161642aba143677e92893e72" candidate_uuid: "4af27de48bec4590bfcd8d71056814eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 34 } } ignore_live_leader: true dest_uuid: "dbc0c6bd96c441e28d5a340d85fbe323"
I20251210 21:05:47.834408 25363 raft_consensus.cc:3055] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 LEADER]: Stepping down as leader of term 6
I20251210 21:05:47.834585 25363 raft_consensus.cc:740] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 128e4ab89fe449e8b5fdc68590b7ac89, State: Running, Role: LEADER
I20251210 21:05:47.834602 25749 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 6 FOLLOWER]: Advancing to term 7
I20251210 21:05:47.834844 25363 consensus_queue.cc:260] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 6.34, Last appended by leader: 34, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:47.835184 25363 raft_consensus.cc:3060] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Advancing to term 7
I20251210 21:05:47.836799 25749 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 7.
I20251210 21:05:47.837106 25363 raft_consensus.cc:2468] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af27de48bec4590bfcd8d71056814eb in term 7.
I20251210 21:05:47.837302 25406 leader_election.cc:304] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af27de48bec4590bfcd8d71056814eb, dbc0c6bd96c441e28d5a340d85fbe323; no voters:
I20251210 21:05:47.837574 26498 raft_consensus.cc:2804] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Leader election won for term 7
I20251210 21:05:47.837975 26498 raft_consensus.cc:697] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 7 LEADER]: Becoming Leader. State: Replica: 4af27de48bec4590bfcd8d71056814eb, State: Running, Role: LEADER
I20251210 21:05:47.838248 26498 consensus_queue.cc:237] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 6.34, Last appended by leader: 34, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } }
I20251210 21:05:47.840751 25258 catalog_manager.cc:5654] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb reported cstate change: term changed from 6 to 7, leader changed from 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65) to 4af27de48bec4590bfcd8d71056814eb (127.21.127.66). New cstate: current_term: 7 leader_uuid: "4af27de48bec4590bfcd8d71056814eb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af27de48bec4590bfcd8d71056814eb" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:48.309258 25749 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 7 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 34. Preceding OpId from leader: term: 7 index: 35. (index mismatch)
I20251210 21:05:48.309839 26498 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc0c6bd96c441e28d5a340d85fbe323" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 37845 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251210 21:05:48.314086 25363 raft_consensus.cc:1275] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 FOLLOWER]: Refusing update from remote peer 4af27de48bec4590bfcd8d71056814eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 34. Preceding OpId from leader: term: 7 index: 35. (index mismatch)
I20251210 21:05:48.314663 26498 consensus_queue.cc:1048] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e4ab89fe449e8b5fdc68590b7ac89" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_rebalancer-test.cc:195: Failure
Expected equality of these values:
0
NumMovesScheduled(leader_idx, BalanceThreadType::LEADER_REBALANCE)
Which is: 1
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_rebalancer-test.cc:575: Failure
Expected: CheckNoLeaderMovesScheduled() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20251210 21:05:48.352584 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:05:48.363384 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:48.363718 22013 tablet_replica.cc:333] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:48.363920 22013 raft_consensus.cc:2243] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.364130 22013 raft_consensus.cc:2272] T 4810894f161642aba143677e92893e72 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.364928 22013 tablet_replica.cc:333] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:48.365095 22013 raft_consensus.cc:2243] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.365525 22013 raft_consensus.cc:2272] T 063c8d31eded49ca9a780697c68d485d P 128e4ab89fe449e8b5fdc68590b7ac89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.366153 22013 tablet_replica.cc:333] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:48.366310 22013 raft_consensus.cc:2243] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.366605 22013 raft_consensus.cc:2272] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 128e4ab89fe449e8b5fdc68590b7ac89 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.367116 22013 tablet_replica.cc:333] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:48.367261 22013 raft_consensus.cc:2243] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 9 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.367578 22013 raft_consensus.cc:2272] T 544270c4ded24ee2ae32f6aa4ec505c2 P 128e4ab89fe449e8b5fdc68590b7ac89 [term 9 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.368083 22013 tablet_replica.cc:333] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:48.368216 22013 raft_consensus.cc:2243] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.368355 22013 raft_consensus.cc:2272] T 05b90891884a4db3bc8723b604cec5ea P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.368815 22013 tablet_replica.cc:333] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89: stopping tablet replica
I20251210 21:05:48.368952 22013 raft_consensus.cc:2243] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.369123 22013 raft_consensus.cc:2272] T b13da827577e49569e304f0cbfb5e9df P 128e4ab89fe449e8b5fdc68590b7ac89 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.406800 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:05:48.412345 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:05:48.422410 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:48.422672 22013 tablet_replica.cc:333] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:48.422838 22013 raft_consensus.cc:2243] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.422983 22013 raft_consensus.cc:2272] T bf7537ffe9f943b3a97e77adab5a1e23 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.423558 22013 tablet_replica.cc:333] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:48.423679 22013 raft_consensus.cc:2243] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.423805 22013 raft_consensus.cc:2272] T b65abf7cc0484fcda919a741e23928a7 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.424237 22013 tablet_replica.cc:333] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:48.424340 22013 raft_consensus.cc:2243] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 7 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.424579 22013 raft_consensus.cc:2272] T 4810894f161642aba143677e92893e72 P 4af27de48bec4590bfcd8d71056814eb [term 7 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.425103 22013 tablet_replica.cc:333] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:48.425200 22013 raft_consensus.cc:2243] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.425338 22013 raft_consensus.cc:2272] T 681d997613474450ba6b9ee3de0f3f24 P 4af27de48bec4590bfcd8d71056814eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.425730 22013 tablet_replica.cc:333] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:48.425827 22013 raft_consensus.cc:2243] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.426049 22013 raft_consensus.cc:2272] T 05b90891884a4db3bc8723b604cec5ea P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.426424 22013 tablet_replica.cc:333] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb: stopping tablet replica
I20251210 21:05:48.426513 22013 raft_consensus.cc:2243] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 6 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.426704 22013 raft_consensus.cc:2272] T b13da827577e49569e304f0cbfb5e9df P 4af27de48bec4590bfcd8d71056814eb [term 6 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.434098 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
W20251210 21:05:48.445706 25479 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.65:41411: connect: Connection refused (error 111)
W20251210 21:05:48.447885 25479 consensus_peers.cc:597] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd -> Peer 128e4ab89fe449e8b5fdc68590b7ac89 (127.21.127.65:41411): Couldn't send request to peer 128e4ab89fe449e8b5fdc68590b7ac89. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:41411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:48.455159 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:05:48.464391 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:48.464656 22013 tablet_replica.cc:333] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:48.464838 22013 raft_consensus.cc:2243] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.465090 22013 raft_consensus.cc:2272] T e7e5750d29e94bad943f054f32b40746 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.465690 22013 tablet_replica.cc:333] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:48.465843 22013 raft_consensus.cc:2243] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.466033 22013 raft_consensus.cc:2272] T 6e20042b566343a7acdbe06f4dcb0c86 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.466507 22013 tablet_replica.cc:333] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:48.466627 22013 raft_consensus.cc:2243] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.466890 22013 raft_consensus.cc:2272] T 063c8d31eded49ca9a780697c68d485d P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.467321 22013 tablet_replica.cc:333] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:48.467458 22013 raft_consensus.cc:2243] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 9 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.467586 22013 raft_consensus.cc:2272] T 544270c4ded24ee2ae32f6aa4ec505c2 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 9 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.468006 22013 tablet_replica.cc:333] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:48.468137 22013 raft_consensus.cc:2243] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.468252 22013 raft_consensus.cc:2272] T 22e5c53fdb9d496b8b87aa938e9a4c3f P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.468650 22013 tablet_replica.cc:333] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd: stopping tablet replica
I20251210 21:05:48.468758 22013 raft_consensus.cc:2243] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.469008 22013 raft_consensus.cc:2272] T b65abf7cc0484fcda919a741e23928a7 P 56e19ce44c0c4f08b1eb440e1ef3e9cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.485317 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:05:48.491264 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:05:48.500218 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:48.500495 22013 tablet_replica.cc:333] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323: stopping tablet replica
I20251210 21:05:48.500671 22013 raft_consensus.cc:2243] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.500916 22013 raft_consensus.cc:2272] T 6e20042b566343a7acdbe06f4dcb0c86 P dbc0c6bd96c441e28d5a340d85fbe323 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.501533 22013 tablet_replica.cc:333] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323: stopping tablet replica
I20251210 21:05:48.501678 22013 raft_consensus.cc:2243] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.501871 22013 raft_consensus.cc:2272] T 063c8d31eded49ca9a780697c68d485d P dbc0c6bd96c441e28d5a340d85fbe323 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.502338 22013 tablet_replica.cc:333] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323: stopping tablet replica
I20251210 21:05:48.502461 22013 raft_consensus.cc:2243] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.502578 22013 raft_consensus.cc:2272] T b65abf7cc0484fcda919a741e23928a7 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.502990 22013 tablet_replica.cc:333] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323: stopping tablet replica
I20251210 21:05:48.503132 22013 raft_consensus.cc:2243] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 7 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.503252 22013 raft_consensus.cc:2272] T 4810894f161642aba143677e92893e72 P dbc0c6bd96c441e28d5a340d85fbe323 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.503918 22013 tablet_replica.cc:333] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323: stopping tablet replica
I20251210 21:05:48.504040 22013 raft_consensus.cc:2243] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.504299 22013 raft_consensus.cc:2272] T bf7537ffe9f943b3a97e77adab5a1e23 P dbc0c6bd96c441e28d5a340d85fbe323 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.504714 22013 tablet_replica.cc:333] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323: stopping tablet replica
I20251210 21:05:48.504843 22013 raft_consensus.cc:2243] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.505088 22013 raft_consensus.cc:2272] T 4f1698d0d3bd4765bc0897771cb0ceee P dbc0c6bd96c441e28d5a340d85fbe323 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.537357 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:05:48.541435 22013 tablet_server.cc:178] TabletServer@127.21.127.69:0 shutting down...
I20251210 21:05:48.550390 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:48.550664 22013 tablet_replica.cc:333] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb: stopping tablet replica
I20251210 21:05:48.550853 22013 raft_consensus.cc:2243] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 6 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.551024 22013 raft_consensus.cc:2272] T b13da827577e49569e304f0cbfb5e9df P a6abef32d32c46478bc3f2b9aa589efb [term 6 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.551628 22013 tablet_replica.cc:333] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb: stopping tablet replica
I20251210 21:05:48.551749 22013 raft_consensus.cc:2243] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 9 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.551951 22013 raft_consensus.cc:2272] T 544270c4ded24ee2ae32f6aa4ec505c2 P a6abef32d32c46478bc3f2b9aa589efb [term 9 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.552383 22013 tablet_replica.cc:333] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb: stopping tablet replica
I20251210 21:05:48.552480 22013 raft_consensus.cc:2243] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.552575 22013 raft_consensus.cc:2272] T 22e5c53fdb9d496b8b87aa938e9a4c3f P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.552970 22013 tablet_replica.cc:333] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb: stopping tablet replica
I20251210 21:05:48.553066 22013 raft_consensus.cc:2243] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.553295 22013 raft_consensus.cc:2272] T 681d997613474450ba6b9ee3de0f3f24 P a6abef32d32c46478bc3f2b9aa589efb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.553674 22013 tablet_replica.cc:333] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb: stopping tablet replica
I20251210 21:05:48.553766 22013 raft_consensus.cc:2243] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.553921 22013 raft_consensus.cc:2272] T 4f1698d0d3bd4765bc0897771cb0ceee P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.554280 22013 tablet_replica.cc:333] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb: stopping tablet replica
I20251210 21:05:48.554368 22013 raft_consensus.cc:2243] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.554567 22013 raft_consensus.cc:2272] T e7e5750d29e94bad943f054f32b40746 P a6abef32d32c46478bc3f2b9aa589efb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.569083 22013 tablet_server.cc:195] TabletServer@127.21.127.69:0 shutdown complete.
I20251210 21:05:48.573100 22013 tablet_server.cc:178] TabletServer@127.21.127.70:0 shutting down...
I20251210 21:05:48.581869 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:48.582154 22013 tablet_replica.cc:333] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094: stopping tablet replica
I20251210 21:05:48.582365 22013 raft_consensus.cc:2243] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.582623 22013 raft_consensus.cc:2272] T 681d997613474450ba6b9ee3de0f3f24 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.583252 22013 tablet_replica.cc:333] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094: stopping tablet replica
I20251210 21:05:48.583387 22013 raft_consensus.cc:2243] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.583599 22013 raft_consensus.cc:2272] T e7e5750d29e94bad943f054f32b40746 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.584100 22013 tablet_replica.cc:333] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094: stopping tablet replica
I20251210 21:05:48.584220 22013 raft_consensus.cc:2243] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.584465 22013 raft_consensus.cc:2272] T 4f1698d0d3bd4765bc0897771cb0ceee P 90c04d3cdd4b4cb18b2a265365b1b094 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.584964 22013 tablet_replica.cc:333] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094: stopping tablet replica
I20251210 21:05:48.585086 22013 raft_consensus.cc:2243] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.585204 22013 raft_consensus.cc:2272] T bf7537ffe9f943b3a97e77adab5a1e23 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.585601 22013 tablet_replica.cc:333] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094: stopping tablet replica
I20251210 21:05:48.585701 22013 raft_consensus.cc:2243] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.585969 22013 raft_consensus.cc:2272] T 6e20042b566343a7acdbe06f4dcb0c86 P 90c04d3cdd4b4cb18b2a265365b1b094 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.586372 22013 tablet_replica.cc:333] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094: stopping tablet replica
I20251210 21:05:48.586475 22013 raft_consensus.cc:2243] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:48.586591 22013 raft_consensus.cc:2272] T 05b90891884a4db3bc8723b604cec5ea P 90c04d3cdd4b4cb18b2a265365b1b094 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.601301 22013 tablet_server.cc:195] TabletServer@127.21.127.70:0 shutdown complete.
I20251210 21:05:48.605516 22013 master.cc:561] Master@127.21.127.126:44841 shutting down...
I20251210 21:05:48.629426 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:05:48.629668 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:48.629768 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P afb8033a8b314114b91c396d6b719bca: stopping tablet replica
I20251210 21:05:48.643432 22013 master.cc:583] Master@127.21.127.126:44841 shutdown complete.
I20251210 21:05:48.650277 22013 test_util.cc:183] -----------------------------------------------
I20251210 21:05:48.650347 22013 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765400617643276-22013-0
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer (37027 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingUnstableCluster
I20251210 21:05:48.652226 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:40181
I20251210 21:05:48.652697 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:48.655427 26530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.655498 26532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.655498 26529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:48.655694 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:48.656111 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:48.656183 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:48.656210 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400748656206 us; error 0 us; skew 500 ppm
I20251210 21:05:48.656453 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:48.657701 22013 webserver.cc:492] Webserver started at http://127.21.127.126:37741/ using document root <none> and password file <none>
I20251210 21:05:48.657922 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:48.657990 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:48.658114 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:48.658725 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "ce915a611aec4880bd29a1ea64ab88fb"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.661228 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:05:48.662782 26537 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.663173 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:48.663272 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "ce915a611aec4880bd29a1ea64ab88fb"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.663371 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:48.670558 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:48.671029 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:48.681730 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:40181
I20251210 21:05:48.681747 26588 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:40181 every 8 connection(s)
I20251210 21:05:48.682996 26589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:48.686995 26589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb: Bootstrap starting.
I20251210 21:05:48.688349 26589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:48.689905 26589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb: No bootstrap required, opened a new log
I20251210 21:05:48.690502 26589 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce915a611aec4880bd29a1ea64ab88fb" member_type: VOTER }
I20251210 21:05:48.690618 26589 raft_consensus.cc:385] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:48.690665 26589 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce915a611aec4880bd29a1ea64ab88fb, State: Initialized, Role: FOLLOWER
I20251210 21:05:48.690840 26589 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce915a611aec4880bd29a1ea64ab88fb" member_type: VOTER }
I20251210 21:05:48.690938 26589 raft_consensus.cc:399] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:05:48.690990 26589 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:05:48.691053 26589 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:48.692814 26589 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce915a611aec4880bd29a1ea64ab88fb" member_type: VOTER }
I20251210 21:05:48.692989 26589 leader_election.cc:304] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce915a611aec4880bd29a1ea64ab88fb; no voters:
I20251210 21:05:48.693347 26589 leader_election.cc:290] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:05:48.693457 26592 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:48.693845 26592 raft_consensus.cc:697] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 1 LEADER]: Becoming Leader. State: Replica: ce915a611aec4880bd29a1ea64ab88fb, State: Running, Role: LEADER
I20251210 21:05:48.694054 26592 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce915a611aec4880bd29a1ea64ab88fb" member_type: VOTER }
I20251210 21:05:48.694131 26589 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:05:48.694733 26593 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce915a611aec4880bd29a1ea64ab88fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce915a611aec4880bd29a1ea64ab88fb" member_type: VOTER } }
I20251210 21:05:48.694790 26594 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce915a611aec4880bd29a1ea64ab88fb. Latest consensus state: current_term: 1 leader_uuid: "ce915a611aec4880bd29a1ea64ab88fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce915a611aec4880bd29a1ea64ab88fb" member_type: VOTER } }
I20251210 21:05:48.695013 26594 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [sys.catalog]: This master's current role is: LEADER
I20251210 21:05:48.695050 26593 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [sys.catalog]: This master's current role is: LEADER
I20251210 21:05:48.695955 26597 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:05:48.697392 26597 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:05:48.698693 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:05:48.700254 26597 catalog_manager.cc:1357] Generated new cluster ID: b8544c86ed734cfe8b288645a6a227c2
I20251210 21:05:48.700330 26597 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:05:48.713032 26597 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:05:48.713812 26597 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:05:48.723760 26597 catalog_manager.cc:6027] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb: Generated new TSK 0
I20251210 21:05:48.724007 26597 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:05:48.731349 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:48.734097 26610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.734171 26611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.734658 26613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:48.734925 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:48.735142 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:48.735193 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:48.735225 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400748735220 us; error 0 us; skew 500 ppm
I20251210 21:05:48.735363 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:48.736531 22013 webserver.cc:492] Webserver started at http://127.21.127.65:45689/ using document root <none> and password file <none>
I20251210 21:05:48.736737 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:48.736799 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:48.736927 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:48.737536 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "730bee39f4e44aedb3e44b8012c9bb3e"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.740013 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:05:48.741488 26618 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.741833 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:05:48.741930 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "730bee39f4e44aedb3e44b8012c9bb3e"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.742030 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:48.768033 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:48.768550 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:48.769284 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:48.770334 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:48.770390 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.770452 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:48.770480 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.783377 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:44953
I20251210 21:05:48.783399 26680 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:44953 every 8 connection(s)
I20251210 21:05:48.784896 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:48.787739 26686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.787827 26685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.788542 26688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:48.790043 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:48.790270 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:48.790323 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:48.790354 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400748790349 us; error 0 us; skew 500 ppm
I20251210 21:05:48.790480 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:48.791779 22013 webserver.cc:492] Webserver started at http://127.21.127.66:38435/ using document root <none> and password file <none>
I20251210 21:05:48.792083 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:48.792183 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:48.792208 26681 heartbeater.cc:344] Connected to a master server at 127.21.127.126:40181
I20251210 21:05:48.792305 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:48.792333 26681 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:48.792630 26681 heartbeater.cc:507] Master 127.21.127.126:40181 requested a full tablet report, sending...
I20251210 21:05:48.792958 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "577fb73919294006aa0c0b6511b5f6f3"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.793531 26554 ts_manager.cc:194] Registered new tserver with Master: 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953)
I20251210 21:05:48.794987 26554 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46592
I20251210 21:05:48.796046 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251210 21:05:48.797552 26693 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.797840 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251210 21:05:48.798017 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "577fb73919294006aa0c0b6511b5f6f3"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.798132 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:48.809238 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:48.809718 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:48.810446 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:48.811260 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:48.811316 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.811388 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:48.811422 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.822906 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:32823
I20251210 21:05:48.822937 26755 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:32823 every 8 connection(s)
I20251210 21:05:48.824360 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:48.826738 26759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.827889 26760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:48.828394 26762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:48.829361 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:48.829595 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:48.829658 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:48.829691 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400748829687 us; error 0 us; skew 500 ppm
I20251210 21:05:48.829826 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:48.830931 26756 heartbeater.cc:344] Connected to a master server at 127.21.127.126:40181
I20251210 21:05:48.831068 26756 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:48.831146 22013 webserver.cc:492] Webserver started at http://127.21.127.67:33229/ using document root <none> and password file <none>
I20251210 21:05:48.831357 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:48.831425 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:48.831458 26756 heartbeater.cc:507] Master 127.21.127.126:40181 requested a full tablet report, sending...
I20251210 21:05:48.831606 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:48.832276 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "2df1f9a30197426bb5b3670743bff531"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.832334 26554 ts_manager.cc:194] Registered new tserver with Master: 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823)
I20251210 21:05:48.833058 26554 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46606
I20251210 21:05:48.834622 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20251210 21:05:48.836247 26767 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.836563 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:48.836655 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "2df1f9a30197426bb5b3670743bff531"
format_stamp: "Formatted at 2025-12-10 21:05:48 on dist-test-slave-9pq8"
I20251210 21:05:48.836752 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:48.858067 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:48.858592 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:48.859136 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:48.860062 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:48.860127 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.860198 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:48.860239 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:48.872610 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:34469
I20251210 21:05:48.872639 26829 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:34469 every 8 connection(s)
I20251210 21:05:48.879081 26830 heartbeater.cc:344] Connected to a master server at 127.21.127.126:40181
I20251210 21:05:48.879217 26830 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:48.879587 26830 heartbeater.cc:507] Master 127.21.127.126:40181 requested a full tablet report, sending...
I20251210 21:05:48.880443 26554 ts_manager.cc:194] Registered new tserver with Master: 2df1f9a30197426bb5b3670743bff531 (127.21.127.67:34469)
I20251210 21:05:48.881204 26554 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46616
I20251210 21:05:48.881438 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007904675s
I20251210 21:05:49.796439 26681 heartbeater.cc:499] Master 127.21.127.126:40181 was elected leader, sending a full tablet report...
I20251210 21:05:49.834245 26756 heartbeater.cc:499] Master 127.21.127.126:40181 was elected leader, sending a full tablet report...
I20251210 21:05:49.882393 26830 heartbeater.cc:499] Master 127.21.127.126:40181 was elected leader, sending a full tablet report...
I20251210 21:05:49.908926 22013 test_util.cc:276] Using random seed: 1687528350
I20251210 21:05:49.921324 26554 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46620:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:05:49.922009 26554 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:05:49.938433 26646 tablet_service.cc:1505] Processing CreateTablet for tablet d88e4a12be14447cb9f5158e67baa97f (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251210 21:05:49.938740 26645 tablet_service.cc:1505] Processing CreateTablet for tablet 0df46524ead843aaa25182bcf7c38ad3 (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251210 21:05:49.938961 26646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d88e4a12be14447cb9f5158e67baa97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.939147 26645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df46524ead843aaa25182bcf7c38ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.939900 26644 tablet_service.cc:1505] Processing CreateTablet for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251210 21:05:49.940317 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.941424 26794 tablet_service.cc:1505] Processing CreateTablet for tablet 0df46524ead843aaa25182bcf7c38ad3 (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251210 21:05:49.941680 26719 tablet_service.cc:1505] Processing CreateTablet for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251210 21:05:49.941778 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df46524ead843aaa25182bcf7c38ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.942042 26719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.943675 26721 tablet_service.cc:1505] Processing CreateTablet for tablet d88e4a12be14447cb9f5158e67baa97f (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251210 21:05:49.944041 26721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d88e4a12be14447cb9f5158e67baa97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.947659 26720 tablet_service.cc:1505] Processing CreateTablet for tablet 0df46524ead843aaa25182bcf7c38ad3 (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251210 21:05:49.947739 26795 tablet_service.cc:1505] Processing CreateTablet for tablet d88e4a12be14447cb9f5158e67baa97f (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251210 21:05:49.948071 26720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df46524ead843aaa25182bcf7c38ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.948108 26795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d88e4a12be14447cb9f5158e67baa97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.951771 26793 tablet_service.cc:1505] Processing CreateTablet for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 (DEFAULT_TABLE table=test-workload [id=1c21c25f63124e2c99f0142d72bab62c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251210 21:05:49.952201 26793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:49.955524 26852 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: Bootstrap starting.
I20251210 21:05:49.958019 26852 tablet_bootstrap.cc:654] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:49.966117 26851 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: Bootstrap starting.
I20251210 21:05:49.966631 26850 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: Bootstrap starting.
I20251210 21:05:49.966872 26852 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: No bootstrap required, opened a new log
I20251210 21:05:49.967025 26852 ts_tablet_manager.cc:1403] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20251210 21:05:49.968113 26851 tablet_bootstrap.cc:654] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:49.968374 26850 tablet_bootstrap.cc:654] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:49.968596 26852 raft_consensus.cc:359] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.968856 26852 raft_consensus.cc:385] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:49.968947 26852 raft_consensus.cc:740] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df1f9a30197426bb5b3670743bff531, State: Initialized, Role: FOLLOWER
I20251210 21:05:49.969344 26852 consensus_queue.cc:260] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.970121 26852 ts_tablet_manager.cc:1434] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20251210 21:05:49.970711 26852 tablet_bootstrap.cc:492] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531: Bootstrap starting.
I20251210 21:05:49.972432 26852 tablet_bootstrap.cc:654] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:49.975543 26852 tablet_bootstrap.cc:492] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531: No bootstrap required, opened a new log
I20251210 21:05:49.975749 26852 ts_tablet_manager.cc:1403] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20251210 21:05:49.976661 26852 raft_consensus.cc:359] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:49.976914 26852 raft_consensus.cc:385] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:49.977013 26852 raft_consensus.cc:740] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df1f9a30197426bb5b3670743bff531, State: Initialized, Role: FOLLOWER
I20251210 21:05:49.977300 26852 consensus_queue.cc:260] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:49.977833 26852 ts_tablet_manager.cc:1434] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:05:49.978194 26852 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531: Bootstrap starting.
I20251210 21:05:49.980042 26852 tablet_bootstrap.cc:654] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:49.982445 26850 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: No bootstrap required, opened a new log
I20251210 21:05:49.982620 26850 ts_tablet_manager.cc:1403] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20251210 21:05:49.983039 26851 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: No bootstrap required, opened a new log
I20251210 21:05:49.983243 26851 ts_tablet_manager.cc:1403] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20251210 21:05:49.983279 26850 raft_consensus.cc:359] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.983464 26850 raft_consensus.cc:385] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:49.983531 26850 raft_consensus.cc:740] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730bee39f4e44aedb3e44b8012c9bb3e, State: Initialized, Role: FOLLOWER
I20251210 21:05:49.983744 26850 consensus_queue.cc:260] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.983994 26851 raft_consensus.cc:359] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.984150 26851 raft_consensus.cc:385] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:49.984241 26851 raft_consensus.cc:740] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577fb73919294006aa0c0b6511b5f6f3, State: Initialized, Role: FOLLOWER
I20251210 21:05:49.984380 26852 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531: No bootstrap required, opened a new log
I20251210 21:05:49.984490 26852 ts_tablet_manager.cc:1403] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20251210 21:05:49.984491 26851 consensus_queue.cc:260] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.985203 26852 raft_consensus.cc:359] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.985358 26852 raft_consensus.cc:385] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:49.985414 26852 raft_consensus.cc:740] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df1f9a30197426bb5b3670743bff531, State: Initialized, Role: FOLLOWER
I20251210 21:05:49.985590 26852 consensus_queue.cc:260] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.986143 26852 ts_tablet_manager.cc:1434] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:05:49.987741 26854 raft_consensus.cc:493] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:49.987872 26854 raft_consensus.cc:515] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:49.988304 26851 ts_tablet_manager.cc:1434] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20251210 21:05:49.989564 26854 leader_election.cc:290] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823)
I20251210 21:05:49.992694 26851 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3: Bootstrap starting.
I20251210 21:05:49.996857 26851 tablet_bootstrap.cc:654] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:49.999783 26859 raft_consensus.cc:493] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:49.999954 26850 ts_tablet_manager.cc:1434] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: Time spent starting tablet: real 0.017s user 0.007s sys 0.010s
I20251210 21:05:49.999984 26859 raft_consensus.cc:515] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.000293 26850 tablet_bootstrap.cc:492] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e: Bootstrap starting.
I20251210 21:05:50.002111 26850 tablet_bootstrap.cc:654] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:50.010318 26850 tablet_bootstrap.cc:492] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e: No bootstrap required, opened a new log
I20251210 21:05:50.010555 26850 ts_tablet_manager.cc:1403] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20251210 21:05:50.010902 26854 raft_consensus.cc:493] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:50.011055 26854 raft_consensus.cc:515] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.011433 26850 raft_consensus.cc:359] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.011617 26850 raft_consensus.cc:385] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:50.011662 26850 raft_consensus.cc:740] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730bee39f4e44aedb3e44b8012c9bb3e, State: Initialized, Role: FOLLOWER
I20251210 21:05:50.011675 26854 leader_election.cc:290] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823)
I20251210 21:05:50.011876 26850 consensus_queue.cc:260] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.012390 26850 ts_tablet_manager.cc:1434] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:05:50.012667 26850 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: Bootstrap starting.
I20251210 21:05:50.014485 26850 tablet_bootstrap.cc:654] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:50.021335 26851 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3: No bootstrap required, opened a new log
I20251210 21:05:50.021961 26851 ts_tablet_manager.cc:1403] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3: Time spent bootstrapping tablet: real 0.029s user 0.012s sys 0.002s
I20251210 21:05:50.022994 26851 raft_consensus.cc:359] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.023285 26851 raft_consensus.cc:385] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:50.023392 26851 raft_consensus.cc:740] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577fb73919294006aa0c0b6511b5f6f3, State: Initialized, Role: FOLLOWER
I20251210 21:05:50.023749 26851 consensus_queue.cc:260] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.027024 26851 ts_tablet_manager.cc:1434] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251210 21:05:50.027163 26859 leader_election.cc:290] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823), 2df1f9a30197426bb5b3670743bff531 (127.21.127.67:34469)
I20251210 21:05:50.028623 26854 raft_consensus.cc:493] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:50.028757 26854 raft_consensus.cc:515] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.029376 26854 leader_election.cc:290] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823)
I20251210 21:05:50.029681 26851 tablet_bootstrap.cc:492] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3: Bootstrap starting.
I20251210 21:05:50.031244 26851 tablet_bootstrap.cc:654] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:50.033419 26851 tablet_bootstrap.cc:492] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3: No bootstrap required, opened a new log
I20251210 21:05:50.033560 26851 ts_tablet_manager.cc:1403] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251210 21:05:50.034386 26851 raft_consensus.cc:359] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.034559 26851 raft_consensus.cc:385] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:50.034639 26851 raft_consensus.cc:740] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577fb73919294006aa0c0b6511b5f6f3, State: Initialized, Role: FOLLOWER
I20251210 21:05:50.034827 26860 raft_consensus.cc:493] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:05:50.034999 26860 raft_consensus.cc:515] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.035070 26851 consensus_queue.cc:260] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.035897 26860 leader_election.cc:290] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 2df1f9a30197426bb5b3670743bff531 (127.21.127.67:34469)
W20251210 21:05:50.036155 26682 tablet.cc:2378] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251210 21:05:50.043499 26731 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3" is_pre_election: true
I20251210 21:05:50.043754 26731 raft_consensus.cc:2468] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 0.
I20251210 21:05:50.044018 26655 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" is_pre_election: true
I20251210 21:05:50.044178 26655 raft_consensus.cc:2468] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 0.
I20251210 21:05:50.044411 26654 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0df46524ead843aaa25182bcf7c38ad3" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" is_pre_election: true
I20251210 21:05:50.044660 26729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0df46524ead843aaa25182bcf7c38ad3" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3" is_pre_election: true
I20251210 21:05:50.044796 26729 raft_consensus.cc:2468] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 0.
I20251210 21:05:50.044975 26730 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3" is_pre_election: true
I20251210 21:05:50.045109 26730 raft_consensus.cc:2468] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 0.
I20251210 21:05:50.045850 26771 leader_election.cc:304] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df1f9a30197426bb5b3670743bff531, 577fb73919294006aa0c0b6511b5f6f3; no voters:
I20251210 21:05:50.046105 26771 leader_election.cc:304] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df1f9a30197426bb5b3670743bff531, 577fb73919294006aa0c0b6511b5f6f3; no voters:
I20251210 21:05:50.046350 26854 raft_consensus.cc:2804] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:50.046417 26854 raft_consensus.cc:493] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:50.046479 26854 raft_consensus.cc:3060] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.046482 26656 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" is_pre_election: true
I20251210 21:05:50.056356 26656 raft_consensus.cc:2468] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 0.
I20251210 21:05:50.061952 26854 raft_consensus.cc:515] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.064268 26875 raft_consensus.cc:2804] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:50.064785 26730 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3" is_pre_election: true
I20251210 21:05:50.064858 26875 raft_consensus.cc:493] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:50.064987 26730 raft_consensus.cc:2468] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730bee39f4e44aedb3e44b8012c9bb3e in term 0.
I20251210 21:05:50.065048 26875 raft_consensus.cc:3060] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.065485 26622 leader_election.cc:304] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 577fb73919294006aa0c0b6511b5f6f3, 730bee39f4e44aedb3e44b8012c9bb3e; no voters:
I20251210 21:05:50.064831 26854 leader_election.cc:290] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 election: Requested vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823)
I20251210 21:05:50.063761 26851 ts_tablet_manager.cc:1434] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3: Time spent starting tablet: real 0.030s user 0.003s sys 0.006s
I20251210 21:05:50.065761 26859 raft_consensus.cc:2804] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:50.065840 26859 raft_consensus.cc:493] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:50.065893 26859 raft_consensus.cc:3060] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.069310 26875 raft_consensus.cc:515] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.069353 26859 raft_consensus.cc:515] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.069919 26875 leader_election.cc:290] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 election: Requested vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823)
I20251210 21:05:50.069948 26859 leader_election.cc:290] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [CANDIDATE]: Term 1 election: Requested vote from peers 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823), 2df1f9a30197426bb5b3670743bff531 (127.21.127.67:34469)
I20251210 21:05:50.070659 26730 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3"
I20251210 21:05:50.062932 26768 leader_election.cc:304] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df1f9a30197426bb5b3670743bff531, 730bee39f4e44aedb3e44b8012c9bb3e; no voters:
I20251210 21:05:50.070873 26730 raft_consensus.cc:3060] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Advancing to term 1
W20251210 21:05:50.071683 26768 leader_election.cc:343] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953): Illegal state: must be running to vote when last-logged opid is not known
I20251210 21:05:50.071712 26729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3"
I20251210 21:05:50.071791 26731 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0df46524ead843aaa25182bcf7c38ad3" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3"
I20251210 21:05:50.071967 26731 raft_consensus.cc:3060] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.072574 26875 raft_consensus.cc:2804] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:05:50.072674 26875 raft_consensus.cc:493] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:05:50.072759 26875 raft_consensus.cc:3060] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.072996 26850 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: No bootstrap required, opened a new log
I20251210 21:05:50.073145 26850 ts_tablet_manager.cc:1403] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: Time spent bootstrapping tablet: real 0.061s user 0.009s sys 0.034s
I20251210 21:05:50.073748 26730 raft_consensus.cc:2468] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730bee39f4e44aedb3e44b8012c9bb3e in term 1.
I20251210 21:05:50.073936 26850 raft_consensus.cc:359] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.074100 26850 raft_consensus.cc:385] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:50.074178 26850 raft_consensus.cc:740] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730bee39f4e44aedb3e44b8012c9bb3e, State: Initialized, Role: FOLLOWER
I20251210 21:05:50.074395 26850 consensus_queue.cc:260] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.074513 26805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0df46524ead843aaa25182bcf7c38ad3" candidate_uuid: "577fb73919294006aa0c0b6511b5f6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df1f9a30197426bb5b3670743bff531" is_pre_election: true
I20251210 21:05:50.074744 26805 raft_consensus.cc:2393] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 577fb73919294006aa0c0b6511b5f6f3 in current term 1: Already voted for candidate 2df1f9a30197426bb5b3670743bff531 in this term.
I20251210 21:05:50.074959 26850 ts_tablet_manager.cc:1434] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:05:50.075060 26654 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0df46524ead843aaa25182bcf7c38ad3" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e"
I20251210 21:05:50.075215 26654 raft_consensus.cc:3060] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.075840 26875 raft_consensus.cc:515] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.076143 26622 leader_election.cc:304] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 577fb73919294006aa0c0b6511b5f6f3, 730bee39f4e44aedb3e44b8012c9bb3e; no voters:
I20251210 21:05:50.076478 26863 raft_consensus.cc:2804] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:50.076624 26863 raft_consensus.cc:697] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 LEADER]: Becoming Leader. State: Replica: 730bee39f4e44aedb3e44b8012c9bb3e, State: Running, Role: LEADER
I20251210 21:05:50.076870 26863 consensus_queue.cc:237] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.077296 26655 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e"
I20251210 21:05:50.077452 26655 raft_consensus.cc:3060] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.077708 26654 raft_consensus.cc:2468] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 1.
I20251210 21:05:50.076864 26730 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb73919294006aa0c0b6511b5f6f3"
I20251210 21:05:50.074963 26656 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "2df1f9a30197426bb5b3670743bff531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e"
I20251210 21:05:50.080132 26655 raft_consensus.cc:2468] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 1.
I20251210 21:05:50.074479 26731 raft_consensus.cc:2468] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 1.
I20251210 21:05:50.078132 26730 raft_consensus.cc:3060] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:50.080884 26771 leader_election.cc:304] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df1f9a30197426bb5b3670743bff531, 577fb73919294006aa0c0b6511b5f6f3; no voters:
I20251210 21:05:50.081184 26854 raft_consensus.cc:2804] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:50.082944 26854 raft_consensus.cc:697] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Becoming Leader. State: Replica: 2df1f9a30197426bb5b3670743bff531, State: Running, Role: LEADER
I20251210 21:05:50.083279 26854 consensus_queue.cc:237] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } }
I20251210 21:05:50.083503 26730 raft_consensus.cc:2468] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df1f9a30197426bb5b3670743bff531 in term 1.
I20251210 21:05:50.083969 26768 leader_election.cc:304] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2df1f9a30197426bb5b3670743bff531; no voters: 577fb73919294006aa0c0b6511b5f6f3, 730bee39f4e44aedb3e44b8012c9bb3e
I20251210 21:05:50.084196 26877 raft_consensus.cc:2749] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251210 21:05:50.084337 26768 leader_election.cc:304] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df1f9a30197426bb5b3670743bff531, 730bee39f4e44aedb3e44b8012c9bb3e; no voters:
I20251210 21:05:50.084618 26877 raft_consensus.cc:2804] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:50.086597 26554 catalog_manager.cc:5654] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2df1f9a30197426bb5b3670743bff531 (127.21.127.67). New cstate: current_term: 1 leader_uuid: "2df1f9a30197426bb5b3670743bff531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:50.088733 26655 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0df46524ead843aaa25182bcf7c38ad3" candidate_uuid: "577fb73919294006aa0c0b6511b5f6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" is_pre_election: true
I20251210 21:05:50.089004 26655 raft_consensus.cc:2393] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 577fb73919294006aa0c0b6511b5f6f3 in current term 1: Already voted for candidate 2df1f9a30197426bb5b3670743bff531 in this term.
I20251210 21:05:50.089180 26875 leader_election.cc:290] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [CANDIDATE]: Term 1 election: Requested vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823)
I20251210 21:05:50.089525 26694 leader_election.cc:304] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 577fb73919294006aa0c0b6511b5f6f3; no voters: 2df1f9a30197426bb5b3670743bff531, 730bee39f4e44aedb3e44b8012c9bb3e
I20251210 21:05:50.089798 26870 raft_consensus.cc:2749] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251210 21:05:50.091204 26804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df1f9a30197426bb5b3670743bff531"
I20251210 21:05:50.091418 26804 raft_consensus.cc:2393] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 730bee39f4e44aedb3e44b8012c9bb3e in current term 1: Already voted for candidate 2df1f9a30197426bb5b3670743bff531 in this term.
I20251210 21:05:50.091164 26805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c8d05dfaa224f27abb2f7c8720a0ca1" candidate_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df1f9a30197426bb5b3670743bff531" is_pre_election: true
I20251210 21:05:50.092231 26805 raft_consensus.cc:2393] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730bee39f4e44aedb3e44b8012c9bb3e in current term 1: Already voted for candidate 2df1f9a30197426bb5b3670743bff531 in this term.
I20251210 21:05:50.092231 26877 raft_consensus.cc:697] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Becoming Leader. State: Replica: 2df1f9a30197426bb5b3670743bff531, State: Running, Role: LEADER
I20251210 21:05:50.093001 26877 consensus_queue.cc:237] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.093940 26554 catalog_manager.cc:5654] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65). New cstate: current_term: 1 leader_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:50.095398 26553 catalog_manager.cc:5654] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2df1f9a30197426bb5b3670743bff531 (127.21.127.67). New cstate: current_term: 1 leader_uuid: "2df1f9a30197426bb5b3670743bff531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } health_report { overall_health: UNKNOWN } } }
I20251210 21:05:50.125689 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:50.128783 26881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:50.128846 26882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:50.137080 26884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:50.137276 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:50.137709 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:50.137787 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:50.137821 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400750137817 us; error 0 us; skew 500 ppm
I20251210 21:05:50.137982 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:50.139268 22013 webserver.cc:492] Webserver started at http://127.21.127.68:33239/ using document root <none> and password file <none>
I20251210 21:05:50.139493 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:50.139557 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:50.139648 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:50.140239 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "369fdc13d98847dca34d2edca7f527c4"
format_stamp: "Formatted at 2025-12-10 21:05:50 on dist-test-slave-9pq8"
I20251210 21:05:50.143539 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20251210 21:05:50.145232 26889 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:50.145772 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:05:50.145879 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "369fdc13d98847dca34d2edca7f527c4"
format_stamp: "Formatted at 2025-12-10 21:05:50 on dist-test-slave-9pq8"
I20251210 21:05:50.146006 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:50.154680 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:50.154949 26863 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:50.155329 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:50.156014 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:50.156930 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:50.156996 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:50.157068 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:50.157104 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20251210 21:05:50.157683 26859 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20251210 21:05:50.157744 26859 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20251210 21:05:50.170321 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:45789
I20251210 21:05:50.170339 26953 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:45789 every 8 connection(s)
I20251210 21:05:50.176738 26954 heartbeater.cc:344] Connected to a master server at 127.21.127.126:40181
I20251210 21:05:50.176877 26954 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:50.177248 26954 heartbeater.cc:507] Master 127.21.127.126:40181 requested a full tablet report, sending...
I20251210 21:05:50.178237 26553 ts_manager.cc:194] Registered new tserver with Master: 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68:45789)
I20251210 21:05:50.178571 26863 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:50.179138 26553 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46624
I20251210 21:05:50.182330 26877 consensus_queue.cc:1048] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:50.188016 26879 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:50.200090 26879 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:50.217841 26879 consensus_queue.cc:1048] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:05:50.698710 26607 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:50.706152 26804 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f"
dest_uuid: "2df1f9a30197426bb5b3670743bff531"
mode: GRACEFUL
new_leader_uuid: "577fb73919294006aa0c0b6511b5f6f3"
from {username='slave'} at 127.0.0.1:46220
I20251210 21:05:50.706385 26804 raft_consensus.cc:606] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Received request to transfer leadership to TS 577fb73919294006aa0c0b6511b5f6f3
I20251210 21:05:50.706813 26607 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251210 21:05:50.706785 26805 consensus_queue.cc:237] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } }
I20251210 21:05:50.707033 26607 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:50.708729 26655 raft_consensus.cc:1275] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Refusing update from remote peer 2df1f9a30197426bb5b3670743bff531: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:50.709718 26730 raft_consensus.cc:1275] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Refusing update from remote peer 2df1f9a30197426bb5b3670743bff531: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:50.709707 26854 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:50.710312 26875 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:50.713027 26730 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
I20251210 21:05:50.712941 26655 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
W20251210 21:05:50.716769 26771 consensus_peers.cc:597] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 -> Peer 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68:45789): Couldn't send request to peer 369fdc13d98847dca34d2edca7f527c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0df46524ead843aaa25182bcf7c38ad3. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:50.717778 26553 catalog_manager.cc:5654] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e reported cstate change: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "2df1f9a30197426bb5b3670743bff531" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
I20251210 21:05:50.790475 26879 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
W20251210 21:05:50.795018 26602 auto_rebalancer.cc:254] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
I20251210 21:05:50.795091 26974 ts_tablet_manager.cc:933] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: Initiating tablet copy from peer 2df1f9a30197426bb5b3670743bff531 (127.21.127.67:34469)
I20251210 21:05:50.795714 26974 tablet_copy_client.cc:323] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:34469
W20251210 21:05:50.797710 26602 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet d88e4a12be14447cb9f5158e67baa97f, TS 577fb73919294006aa0c0b6511b5f6f3 -> TS 369fdc13d98847dca34d2edca7f527c4 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
W20251210 21:05:50.797797 26602 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet d88e4a12be14447cb9f5158e67baa97f, TS 577fb73919294006aa0c0b6511b5f6f3 -> TS 369fdc13d98847dca34d2edca7f527c4 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.800967 26815 tablet_copy_service.cc:140] P 2df1f9a30197426bb5b3670743bff531: Received BeginTabletCopySession request for tablet 0df46524ead843aaa25182bcf7c38ad3 from peer 369fdc13d98847dca34d2edca7f527c4 ({username='slave'} at 127.0.0.1:46244)
I20251210 21:05:50.801138 26815 tablet_copy_service.cc:161] P 2df1f9a30197426bb5b3670743bff531: Beginning new tablet copy session on tablet 0df46524ead843aaa25182bcf7c38ad3 from peer 369fdc13d98847dca34d2edca7f527c4 at {username='slave'} at 127.0.0.1:46244: session id = 369fdc13d98847dca34d2edca7f527c4-0df46524ead843aaa25182bcf7c38ad3
I20251210 21:05:50.803788 26815 tablet_copy_source_session.cc:215] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:50.805039 26974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df46524ead843aaa25182bcf7c38ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:50.805675 26877 raft_consensus.cc:993] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531: : Instructing follower 577fb73919294006aa0c0b6511b5f6f3 to start an election
I20251210 21:05:50.805845 26877 raft_consensus.cc:1081] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Signalling peer 577fb73919294006aa0c0b6511b5f6f3 to start an election
I20251210 21:05:50.806435 26730 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f"
dest_uuid: "577fb73919294006aa0c0b6511b5f6f3"
from {username='slave'} at 127.0.0.1:53508
I20251210 21:05:50.806628 26730 raft_consensus.cc:493] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251210 21:05:50.806753 26730 raft_consensus.cc:3060] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:50.809077 26730 raft_consensus.cc:515] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.810084 26804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f" candidate_uuid: "577fb73919294006aa0c0b6511b5f6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2df1f9a30197426bb5b3670743bff531"
I20251210 21:05:50.810084 26655 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d88e4a12be14447cb9f5158e67baa97f" candidate_uuid: "577fb73919294006aa0c0b6511b5f6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730bee39f4e44aedb3e44b8012c9bb3e"
I20251210 21:05:50.810309 26655 raft_consensus.cc:3060] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:50.810365 26730 leader_election.cc:290] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953), 2df1f9a30197426bb5b3670743bff531 (127.21.127.67:34469)
I20251210 21:05:50.810612 26974 tablet_copy_client.cc:806] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:50.810773 26974 tablet_copy_client.cc:670] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:50.812067 26974 tablet_copy_client.cc:538] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:50.812150 26655 raft_consensus.cc:2468] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 577fb73919294006aa0c0b6511b5f6f3 in term 2.
I20251210 21:05:50.810292 26804 raft_consensus.cc:3055] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Stepping down as leader of term 1
I20251210 21:05:50.812485 26804 raft_consensus.cc:740] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2df1f9a30197426bb5b3670743bff531, State: Running, Role: LEADER
I20251210 21:05:50.812517 26694 leader_election.cc:304] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 577fb73919294006aa0c0b6511b5f6f3, 730bee39f4e44aedb3e44b8012c9bb3e; no voters:
I20251210 21:05:50.812711 26804 consensus_queue.cc:260] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.812798 26870 raft_consensus.cc:2804] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:05:50.812937 26804 raft_consensus.cc:3060] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:05:50.813289 26870 raft_consensus.cc:697] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 577fb73919294006aa0c0b6511b5f6f3, State: Running, Role: LEADER
I20251210 21:05:50.813555 26870 consensus_queue.cc:237] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } }
I20251210 21:05:50.814792 26804 raft_consensus.cc:2468] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 577fb73919294006aa0c0b6511b5f6f3 in term 2.
I20251210 21:05:50.815364 26974 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: Bootstrap starting.
I20251210 21:05:50.820955 26974 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:50.821372 26974 tablet_bootstrap.cc:492] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: Bootstrap complete.
I20251210 21:05:50.821645 26974 ts_tablet_manager.cc:1403] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.001s
I20251210 21:05:50.822479 26974 raft_consensus.cc:359] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } }
I20251210 21:05:50.822705 26974 raft_consensus.cc:740] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 369fdc13d98847dca34d2edca7f527c4, State: Initialized, Role: LEARNER
I20251210 21:05:50.822921 26974 consensus_queue.cc:260] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } }
I20251210 21:05:50.822971 26552 catalog_manager.cc:5654] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 reported cstate change: term changed from 1 to 2, leader changed from 2df1f9a30197426bb5b3670743bff531 (127.21.127.67) to 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66). New cstate: current_term: 2 leader_uuid: "577fb73919294006aa0c0b6511b5f6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } health_report { overall_health: HEALTHY } } }
I20251210 21:05:50.824856 26954 heartbeater.cc:499] Master 127.21.127.126:40181 was elected leader, sending a full tablet report...
I20251210 21:05:50.825356 26974 ts_tablet_manager.cc:1434] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251210 21:05:50.825948 26815 tablet_copy_service.cc:342] P 2df1f9a30197426bb5b3670743bff531: Request end of tablet copy session 369fdc13d98847dca34d2edca7f527c4-0df46524ead843aaa25182bcf7c38ad3 received from {username='slave'} at 127.0.0.1:46244
I20251210 21:05:50.826086 26815 tablet_copy_service.cc:434] P 2df1f9a30197426bb5b3670743bff531: ending tablet copy session 369fdc13d98847dca34d2edca7f527c4-0df46524ead843aaa25182bcf7c38ad3 on tablet 0df46524ead843aaa25182bcf7c38ad3 with peer 369fdc13d98847dca34d2edca7f527c4
I20251210 21:05:50.895740 26929 raft_consensus.cc:1217] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251210 21:05:50.915506 26655 raft_consensus.cc:1275] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 2 FOLLOWER]: Refusing update from remote peer 577fb73919294006aa0c0b6511b5f6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:05:50.916076 26870 consensus_queue.cc:1048] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:50.933738 26804 raft_consensus.cc:1275] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 2 FOLLOWER]: Refusing update from remote peer 577fb73919294006aa0c0b6511b5f6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:05:50.934211 26977 consensus_queue.cc:1048] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:51.030797 26973 raft_consensus.cc:1064] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: attempting to promote NON_VOTER 369fdc13d98847dca34d2edca7f527c4 to VOTER
I20251210 21:05:51.031572 26973 consensus_queue.cc:237] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } }
I20251210 21:05:51.033334 26730 raft_consensus.cc:1275] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Refusing update from remote peer 2df1f9a30197426bb5b3670743bff531: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:05:51.033339 26655 raft_consensus.cc:1275] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Refusing update from remote peer 2df1f9a30197426bb5b3670743bff531: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:05:51.033617 26929 raft_consensus.cc:1275] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Refusing update from remote peer 2df1f9a30197426bb5b3670743bff531: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:05:51.033845 26879 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:05:51.034755 26973 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:05:51.035027 26877 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:05:51.038568 26929 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.039719 26730 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.043109 26553 catalog_manager.cc:5654] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 reported cstate change: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2df1f9a30197426bb5b3670743bff531" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.068982 26879 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.072362 26655 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } attrs { replace: true } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.134215 26804 consensus_queue.cc:237] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } }
I20251210 21:05:51.135955 26929 raft_consensus.cc:1275] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Refusing update from remote peer 2df1f9a30197426bb5b3670743bff531: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:05:51.135989 26730 raft_consensus.cc:1275] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Refusing update from remote peer 2df1f9a30197426bb5b3670743bff531: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:05:51.136458 26875 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:05:51.136771 26973 consensus_queue.cc:1048] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:05:51.139392 26730 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.142061 26929 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.145597 26554 catalog_manager.cc:5654] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 reported cstate change: config changed from index 3 to 4, VOTER 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65) evicted. New cstate: current_term: 1 leader_uuid: "2df1f9a30197426bb5b3670743bff531" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.165221 26644 tablet_service.cc:1552] Processing DeleteTablet for tablet 0df46524ead843aaa25182bcf7c38ad3 with delete_type TABLET_DATA_TOMBSTONED (TS 730bee39f4e44aedb3e44b8012c9bb3e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56882
I20251210 21:05:51.165853 26986 tablet_replica.cc:333] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: stopping tablet replica
I20251210 21:05:51.166062 26986 raft_consensus.cc:2243] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:51.166332 26986 raft_consensus.cc:2272] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:51.167117 26986 ts_tablet_manager.cc:1916] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:51.171957 26986 ts_tablet_manager.cc:1929] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251210 21:05:51.172091 26986 log.cc:1199] T 0df46524ead843aaa25182bcf7c38ad3 P 730bee39f4e44aedb3e44b8012c9bb3e: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/0df46524ead843aaa25182bcf7c38ad3
I20251210 21:05:51.172706 26538 catalog_manager.cc:4985] TS 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953): tablet 0df46524ead843aaa25182bcf7c38ad3 (table test-workload [id=1c21c25f63124e2c99f0142d72bab62c]) successfully deleted
I20251210 21:05:51.269656 26879 raft_consensus.cc:2955] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:51.272432 26541 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 0df46524ead843aaa25182bcf7c38ad3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:52.281764 26656 consensus_queue.cc:237] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } }
I20251210 21:05:52.284907 26730 raft_consensus.cc:1275] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Refusing update from remote peer 730bee39f4e44aedb3e44b8012c9bb3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:52.284945 26804 raft_consensus.cc:1275] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Refusing update from remote peer 730bee39f4e44aedb3e44b8012c9bb3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:05:52.285466 26990 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:52.285789 26859 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:05:52.289391 26730 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
W20251210 21:05:52.290668 26622 consensus_peers.cc:597] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e -> Peer 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68:45789): Couldn't send request to peer 369fdc13d98847dca34d2edca7f527c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c8d05dfaa224f27abb2f7c8720a0ca1. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:52.293448 26554 catalog_manager.cc:5654] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
I20251210 21:05:52.298520 26804 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
I20251210 21:05:52.318434 26990 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } } }
I20251210 21:05:52.404457 27001 ts_tablet_manager.cc:933] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: Initiating tablet copy from peer 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953)
I20251210 21:05:52.405279 27001 tablet_copy_client.cc:323] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:44953
I20251210 21:05:52.411689 26666 tablet_copy_service.cc:140] P 730bee39f4e44aedb3e44b8012c9bb3e: Received BeginTabletCopySession request for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 from peer 369fdc13d98847dca34d2edca7f527c4 ({username='slave'} at 127.0.0.1:56930)
I20251210 21:05:52.411837 26666 tablet_copy_service.cc:161] P 730bee39f4e44aedb3e44b8012c9bb3e: Beginning new tablet copy session on tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 from peer 369fdc13d98847dca34d2edca7f527c4 at {username='slave'} at 127.0.0.1:56930: session id = 369fdc13d98847dca34d2edca7f527c4-9c8d05dfaa224f27abb2f7c8720a0ca1
I20251210 21:05:52.414175 26666 tablet_copy_source_session.cc:215] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:05:52.415210 27001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:52.419981 27001 tablet_copy_client.cc:806] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Starting download of 0 data blocks...
I20251210 21:05:52.420265 27001 tablet_copy_client.cc:670] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Starting download of 1 WAL segments...
I20251210 21:05:52.421589 27001 tablet_copy_client.cc:538] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:05:52.424506 27001 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: Bootstrap starting.
I20251210 21:05:52.430225 27001 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:05:52.430650 27001 tablet_bootstrap.cc:492] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: Bootstrap complete.
I20251210 21:05:52.430966 27001 ts_tablet_manager.cc:1403] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20251210 21:05:52.431728 27001 raft_consensus.cc:359] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } }
I20251210 21:05:52.431950 27001 raft_consensus.cc:740] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 369fdc13d98847dca34d2edca7f527c4, State: Initialized, Role: LEARNER
I20251210 21:05:52.432129 27001 consensus_queue.cc:260] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: true } }
I20251210 21:05:52.432837 27001 ts_tablet_manager.cc:1434] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:05:52.433418 26666 tablet_copy_service.cc:342] P 730bee39f4e44aedb3e44b8012c9bb3e: Request end of tablet copy session 369fdc13d98847dca34d2edca7f527c4-9c8d05dfaa224f27abb2f7c8720a0ca1 received from {username='slave'} at 127.0.0.1:56930
I20251210 21:05:52.433531 26666 tablet_copy_service.cc:434] P 730bee39f4e44aedb3e44b8012c9bb3e: ending tablet copy session 369fdc13d98847dca34d2edca7f527c4-9c8d05dfaa224f27abb2f7c8720a0ca1 on tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 with peer 369fdc13d98847dca34d2edca7f527c4
I20251210 21:05:52.501549 26929 raft_consensus.cc:1217] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251210 21:05:52.626964 26990 raft_consensus.cc:1064] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: attempting to promote NON_VOTER 369fdc13d98847dca34d2edca7f527c4 to VOTER
I20251210 21:05:52.627826 26990 consensus_queue.cc:237] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } }
I20251210 21:05:52.629518 26929 raft_consensus.cc:1275] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 LEARNER]: Refusing update from remote peer 730bee39f4e44aedb3e44b8012c9bb3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:05:52.629573 26804 raft_consensus.cc:1275] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Refusing update from remote peer 730bee39f4e44aedb3e44b8012c9bb3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:05:52.630014 26859 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:05:52.630115 26730 raft_consensus.cc:1275] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Refusing update from remote peer 730bee39f4e44aedb3e44b8012c9bb3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:05:52.630308 26999 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:05:52.631036 26999 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:05:52.633565 26929 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.633780 26730 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.638370 26551 catalog_manager.cc:5654] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 reported cstate change: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.693140 26859 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.696492 26804 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 369fdc13d98847dca34d2edca7f527c4 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "577fb73919294006aa0c0b6511b5f6f3" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 32823 } attrs { replace: true } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.698890 26656 consensus_queue.cc:237] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } }
I20251210 21:05:52.701697 26929 raft_consensus.cc:1275] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Refusing update from remote peer 730bee39f4e44aedb3e44b8012c9bb3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:05:52.701697 26804 raft_consensus.cc:1275] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Refusing update from remote peer 730bee39f4e44aedb3e44b8012c9bb3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:05:52.702253 26990 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:05:52.702481 27007 consensus_queue.cc:1048] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:05:52.705175 26929 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.707370 26607 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:52.708097 26607 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:52.708235 26607 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:52.708895 26804 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.709928 26552 catalog_manager.cc:5654] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 reported cstate change: config changed from index 3 to 4, VOTER 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66) evicted. New cstate: current_term: 1 leader_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.783898 27007 raft_consensus.cc:2955] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730bee39f4e44aedb3e44b8012c9bb3e" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44953 } } peers { permanent_uuid: "2df1f9a30197426bb5b3670743bff531" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 34469 } } peers { permanent_uuid: "369fdc13d98847dca34d2edca7f527c4" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 45789 } attrs { promote: false } } }
I20251210 21:05:52.792313 26538 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:05:52.793888 26719 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 with delete_type TABLET_DATA_TOMBSTONED (TS 577fb73919294006aa0c0b6511b5f6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53502
I20251210 21:05:52.796183 27008 tablet_replica.cc:333] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: stopping tablet replica
I20251210 21:05:52.796454 27008 raft_consensus.cc:2243] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:52.796844 27008 raft_consensus.cc:2272] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:52.798028 27008 ts_tablet_manager.cc:1916] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:05:52.808295 27008 ts_tablet_manager.cc:1929] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251210 21:05:52.808400 27008 log.cc:1199] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 577fb73919294006aa0c0b6511b5f6f3: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/9c8d05dfaa224f27abb2f7c8720a0ca1
I20251210 21:05:52.808980 26541 catalog_manager.cc:4985] TS 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823): tablet 9c8d05dfaa224f27abb2f7c8720a0ca1 (table test-workload [id=1c21c25f63124e2c99f0142d72bab62c]) successfully deleted
I20251210 21:05:54.708590 26607 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:54.709344 26607 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:54.709462 26607 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:56.709932 26607 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:56.710739 26607 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:56.710884 26607 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:58.711258 26607 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:05:58.712141 26607 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:05:58.712296 26607 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:05:59.231120 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:05:59.258988 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:59.259285 22013 tablet_replica.cc:333] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e: stopping tablet replica
I20251210 21:05:59.259521 22013 raft_consensus.cc:2243] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:05:59.259881 22013 raft_consensus.cc:2272] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 730bee39f4e44aedb3e44b8012c9bb3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.260491 22013 tablet_replica.cc:333] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e: stopping tablet replica
I20251210 21:05:59.260619 22013 raft_consensus.cc:2243] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:59.260782 22013 raft_consensus.cc:2272] T d88e4a12be14447cb9f5158e67baa97f P 730bee39f4e44aedb3e44b8012c9bb3e [term 2 FOLLOWER]: Raft consensus is shut down!
W20251210 21:05:59.263744 26694 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:05:59.264869 26694 connection.cc:537] client connection to 127.21.127.65:44953 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251210 21:05:59.265057 26694 consensus_peers.cc:597] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 -> Peer 730bee39f4e44aedb3e44b8012c9bb3e (127.21.127.65:44953): Couldn't send request to peer 730bee39f4e44aedb3e44b8012c9bb3e. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:59.266579 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:05:59.271493 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:05:59.299732 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:59.300041 22013 tablet_replica.cc:333] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3: stopping tablet replica
I20251210 21:05:59.300252 22013 raft_consensus.cc:2243] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:05:59.300519 22013 raft_consensus.cc:2272] T d88e4a12be14447cb9f5158e67baa97f P 577fb73919294006aa0c0b6511b5f6f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.301076 22013 tablet_replica.cc:333] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3: stopping tablet replica
I20251210 21:05:59.301178 22013 raft_consensus.cc:2243] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:59.301378 22013 raft_consensus.cc:2272] T 0df46524ead843aaa25182bcf7c38ad3 P 577fb73919294006aa0c0b6511b5f6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251210 21:05:59.309581 26771 consensus_peers.cc:597] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 -> Peer 577fb73919294006aa0c0b6511b5f6f3 (127.21.127.66:32823): Couldn't send request to peer 577fb73919294006aa0c0b6511b5f6f3. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.66:32823: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:05:59.315459 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:05:59.318807 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:05:59.327876 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:59.328161 22013 tablet_replica.cc:333] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531: stopping tablet replica
I20251210 21:05:59.328358 22013 raft_consensus.cc:2243] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:59.328603 22013 raft_consensus.cc:2272] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.329247 22013 tablet_replica.cc:333] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531: stopping tablet replica
I20251210 21:05:59.329365 22013 raft_consensus.cc:2243] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:05:59.329628 22013 raft_consensus.cc:2272] T 0df46524ead843aaa25182bcf7c38ad3 P 2df1f9a30197426bb5b3670743bff531 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.330076 22013 tablet_replica.cc:333] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531: stopping tablet replica
I20251210 21:05:59.330180 22013 raft_consensus.cc:2243] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:59.330283 22013 raft_consensus.cc:2272] T d88e4a12be14447cb9f5158e67baa97f P 2df1f9a30197426bb5b3670743bff531 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.346961 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:05:59.350688 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:05:59.359426 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:05:59.359737 22013 tablet_replica.cc:333] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4: stopping tablet replica
I20251210 21:05:59.359920 22013 raft_consensus.cc:2243] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:59.360195 22013 raft_consensus.cc:2272] T 9c8d05dfaa224f27abb2f7c8720a0ca1 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.360816 22013 tablet_replica.cc:333] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4: stopping tablet replica
I20251210 21:05:59.360946 22013 raft_consensus.cc:2243] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:05:59.361177 22013 raft_consensus.cc:2272] T 0df46524ead843aaa25182bcf7c38ad3 P 369fdc13d98847dca34d2edca7f527c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.375182 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:05:59.378584 22013 master.cc:561] Master@127.21.127.126:40181 shutting down...
I20251210 21:05:59.387363 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:05:59.387647 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:05:59.387739 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P ce915a611aec4880bd29a1ea64ab88fb: stopping tablet replica
I20251210 21:05:59.401278 22013 master.cc:583] Master@127.21.127.126:40181 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingUnstableCluster (10784 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor
I20251210 21:05:59.436437 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:38205
I20251210 21:05:59.436990 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:59.439723 27017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:59.439708 22013 server_base.cc:1047] running on GCE node
W20251210 21:05:59.439713 27014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:59.439684 27015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:59.440343 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:59.440400 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:59.440429 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400759440425 us; error 0 us; skew 500 ppm
I20251210 21:05:59.440568 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:59.441648 22013 webserver.cc:492] Webserver started at http://127.21.127.126:40553/ using document root <none> and password file <none>
I20251210 21:05:59.441864 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:59.441923 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:59.442062 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:59.442678 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "4be93477b3924c41a22e5c00906a576b"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.445202 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251210 21:05:59.446722 27022 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.447127 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:05:59.447220 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "4be93477b3924c41a22e5c00906a576b"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.447316 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:59.461311 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:59.461846 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:59.473150 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:38205
I20251210 21:05:59.473170 27073 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:38205 every 8 connection(s)
I20251210 21:05:59.474447 27074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:05:59.478538 27074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b: Bootstrap starting.
I20251210 21:05:59.479902 27074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b: Neither blocks nor log segments found. Creating new log.
I20251210 21:05:59.481812 27074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b: No bootstrap required, opened a new log
I20251210 21:05:59.482471 27074 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be93477b3924c41a22e5c00906a576b" member_type: VOTER }
I20251210 21:05:59.482604 27074 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:05:59.482656 27074 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be93477b3924c41a22e5c00906a576b, State: Initialized, Role: FOLLOWER
I20251210 21:05:59.482838 27074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be93477b3924c41a22e5c00906a576b" member_type: VOTER }
I20251210 21:05:59.482956 27074 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:05:59.483032 27074 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:05:59.483119 27074 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:05:59.485021 27074 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be93477b3924c41a22e5c00906a576b" member_type: VOTER }
I20251210 21:05:59.485247 27074 leader_election.cc:304] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4be93477b3924c41a22e5c00906a576b; no voters:
I20251210 21:05:59.485703 27074 leader_election.cc:290] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:05:59.485852 27077 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:05:59.486267 27077 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 1 LEADER]: Becoming Leader. State: Replica: 4be93477b3924c41a22e5c00906a576b, State: Running, Role: LEADER
I20251210 21:05:59.486553 27074 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:05:59.486515 27077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be93477b3924c41a22e5c00906a576b" member_type: VOTER }
I20251210 21:05:59.487792 27079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4be93477b3924c41a22e5c00906a576b. Latest consensus state: current_term: 1 leader_uuid: "4be93477b3924c41a22e5c00906a576b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be93477b3924c41a22e5c00906a576b" member_type: VOTER } }
I20251210 21:05:59.488004 27079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [sys.catalog]: This master's current role is: LEADER
I20251210 21:05:59.488497 27078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4be93477b3924c41a22e5c00906a576b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be93477b3924c41a22e5c00906a576b" member_type: VOTER } }
I20251210 21:05:59.488734 27078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [sys.catalog]: This master's current role is: LEADER
I20251210 21:05:59.488893 27084 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:05:59.490404 27084 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:05:59.493993 27084 catalog_manager.cc:1357] Generated new cluster ID: 4b4df3e8b02e423e99bac013be7a95e2
I20251210 21:05:59.494133 27084 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:05:59.494166 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:05:59.501426 27084 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:05:59.502292 27084 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:05:59.509133 27084 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b: Generated new TSK 0
I20251210 21:05:59.509387 27084 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:05:59.526950 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:59.529930 27095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:59.529958 27096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:59.530231 27098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:59.530588 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:59.530962 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:59.531025 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:59.531067 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400759531052 us; error 0 us; skew 500 ppm
I20251210 21:05:59.531196 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:59.532363 22013 webserver.cc:492] Webserver started at http://127.21.127.65:37611/ using document root <none> and password file <none>
I20251210 21:05:59.532577 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:59.532645 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:59.532754 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:59.533397 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "c25e6a66028540a599247b4aaddb4150"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.535864 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:05:59.537439 27103 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.537771 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:05:59.537882 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "c25e6a66028540a599247b4aaddb4150"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.538015 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:59.548677 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:59.549266 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:59.549880 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:59.550755 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:59.550827 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.550925 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:59.550987 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.563050 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:44929
I20251210 21:05:59.563071 27165 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:44929 every 8 connection(s)
I20251210 21:05:59.564626 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:59.567433 27171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:59.567480 27170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:59.568073 27173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:59.571060 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:59.571381 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:59.571496 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:59.571558 27166 heartbeater.cc:344] Connected to a master server at 127.21.127.126:38205
I20251210 21:05:59.571568 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400759571542 us; error 0 us; skew 500 ppm
I20251210 21:05:59.571727 27166 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:59.571779 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:59.572135 27166 heartbeater.cc:507] Master 127.21.127.126:38205 requested a full tablet report, sending...
I20251210 21:05:59.573282 27039 ts_manager.cc:194] Registered new tserver with Master: c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929)
I20251210 21:05:59.573349 22013 webserver.cc:492] Webserver started at http://127.21.127.66:45575/ using document root <none> and password file <none>
I20251210 21:05:59.573568 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:59.573644 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:59.573779 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:59.574391 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "45175aad53214b239a2976a1ff066dcf"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.574450 27039 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34996
I20251210 21:05:59.576838 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:05:59.578429 27178 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.578801 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:05:59.578898 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "45175aad53214b239a2976a1ff066dcf"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.579022 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:59.588351 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:59.588930 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:59.589468 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:59.590266 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:59.590335 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.590435 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:59.590487 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.603629 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:43227
I20251210 21:05:59.603645 27240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:43227 every 8 connection(s)
I20251210 21:05:59.605158 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:05:59.607985 27244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:59.611690 27245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:05:59.612188 27247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:05:59.612202 22013 server_base.cc:1047] running on GCE node
I20251210 21:05:59.612356 27241 heartbeater.cc:344] Connected to a master server at 127.21.127.126:38205
I20251210 21:05:59.612531 27241 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:59.612612 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:05:59.612696 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:05:59.612757 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400759612735 us; error 0 us; skew 500 ppm
I20251210 21:05:59.612911 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:05:59.612927 27241 heartbeater.cc:507] Master 127.21.127.126:38205 requested a full tablet report, sending...
I20251210 21:05:59.614022 27039 ts_manager.cc:194] Registered new tserver with Master: 45175aad53214b239a2976a1ff066dcf (127.21.127.66:43227)
I20251210 21:05:59.614243 22013 webserver.cc:492] Webserver started at http://127.21.127.67:34285/ using document root <none> and password file <none>
I20251210 21:05:59.614476 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:05:59.614569 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:05:59.614706 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:05:59.615044 27039 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35012
I20251210 21:05:59.615504 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.617756 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:05:59.619262 27252 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.619590 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:05:59.619690 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429"
format_stamp: "Formatted at 2025-12-10 21:05:59 on dist-test-slave-9pq8"
I20251210 21:05:59.619808 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:05:59.647394 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:05:59.647987 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:05:59.648563 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:05:59.649446 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:05:59.649514 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.649607 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:05:59.649662 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:05:59.662325 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:35535
I20251210 21:05:59.662340 27314 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:35535 every 8 connection(s)
I20251210 21:05:59.668751 27315 heartbeater.cc:344] Connected to a master server at 127.21.127.126:38205
I20251210 21:05:59.668972 27315 heartbeater.cc:461] Registering TS with master...
I20251210 21:05:59.669399 27315 heartbeater.cc:507] Master 127.21.127.126:38205 requested a full tablet report, sending...
I20251210 21:05:59.670274 27039 ts_manager.cc:194] Registered new tserver with Master: 4d5cbf0a0edc4610a2115a8e6c5d2429 (127.21.127.67:35535)
I20251210 21:05:59.671041 27039 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35014
I20251210 21:05:59.671065 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007911306s
I20251210 21:06:00.575807 27166 heartbeater.cc:499] Master 127.21.127.126:38205 was elected leader, sending a full tablet report...
I20251210 21:06:00.616390 27241 heartbeater.cc:499] Master 127.21.127.126:38205 was elected leader, sending a full tablet report...
I20251210 21:06:00.672653 27315 heartbeater.cc:499] Master 127.21.127.126:38205 was elected leader, sending a full tablet report...
I20251210 21:06:00.698803 22013 test_util.cc:276] Using random seed: 1698318238
I20251210 21:06:00.711679 27039 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35018:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:06:00.712342 27039 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:06:00.740686 27206 tablet_service.cc:1505] Processing CreateTablet for tablet 80d9013ef87e487684fce4bd2e081c3c (DEFAULT_TABLE table=test-workload [id=83200e3b9ace4df1b9a0538bf7a565fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:06:00.741214 27206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d9013ef87e487684fce4bd2e081c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:00.742214 27280 tablet_service.cc:1505] Processing CreateTablet for tablet 80d9013ef87e487684fce4bd2e081c3c (DEFAULT_TABLE table=test-workload [id=83200e3b9ace4df1b9a0538bf7a565fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:06:00.742558 27280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d9013ef87e487684fce4bd2e081c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:00.743319 27131 tablet_service.cc:1505] Processing CreateTablet for tablet 80d9013ef87e487684fce4bd2e081c3c (DEFAULT_TABLE table=test-workload [id=83200e3b9ace4df1b9a0538bf7a565fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251210 21:06:00.743752 27131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d9013ef87e487684fce4bd2e081c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:00.749351 27335 tablet_bootstrap.cc:492] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429: Bootstrap starting.
I20251210 21:06:00.750974 27335 tablet_bootstrap.cc:654] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:00.751921 27337 tablet_bootstrap.cc:492] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150: Bootstrap starting.
I20251210 21:06:00.752802 27336 tablet_bootstrap.cc:492] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf: Bootstrap starting.
I20251210 21:06:00.753993 27337 tablet_bootstrap.cc:654] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:00.754648 27336 tablet_bootstrap.cc:654] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:00.755081 27335 tablet_bootstrap.cc:492] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429: No bootstrap required, opened a new log
I20251210 21:06:00.755255 27335 ts_tablet_manager.cc:1403] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251210 21:06:00.755949 27337 tablet_bootstrap.cc:492] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150: No bootstrap required, opened a new log
I20251210 21:06:00.756100 27337 ts_tablet_manager.cc:1403] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:00.756214 27335 raft_consensus.cc:359] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.756426 27335 raft_consensus.cc:385] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:00.756460 27336 tablet_bootstrap.cc:492] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf: No bootstrap required, opened a new log
I20251210 21:06:00.756521 27335 raft_consensus.cc:740] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5cbf0a0edc4610a2115a8e6c5d2429, State: Initialized, Role: FOLLOWER
I20251210 21:06:00.756641 27336 ts_tablet_manager.cc:1403] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20251210 21:06:00.756778 27335 consensus_queue.cc:260] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.756892 27337 raft_consensus.cc:359] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.757091 27337 raft_consensus.cc:385] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:00.757252 27337 raft_consensus.cc:740] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25e6a66028540a599247b4aaddb4150, State: Initialized, Role: FOLLOWER
I20251210 21:06:00.757357 27336 raft_consensus.cc:359] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.757603 27336 raft_consensus.cc:385] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:00.757582 27335 ts_tablet_manager.cc:1434] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251210 21:06:00.757745 27336 raft_consensus.cc:740] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45175aad53214b239a2976a1ff066dcf, State: Initialized, Role: FOLLOWER
I20251210 21:06:00.757565 27337 consensus_queue.cc:260] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.758159 27336 consensus_queue.cc:260] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.758581 27337 ts_tablet_manager.cc:1434] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:06:00.763933 27336 ts_tablet_manager.cc:1434] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20251210 21:06:00.771200 27341 raft_consensus.cc:493] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:00.771382 27341 raft_consensus.cc:515] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.772292 27341 leader_election.cc:290] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45175aad53214b239a2976a1ff066dcf (127.21.127.66:43227), c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929)
I20251210 21:06:00.778097 27141 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d9013ef87e487684fce4bd2e081c3c" candidate_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25e6a66028540a599247b4aaddb4150" is_pre_election: true
I20251210 21:06:00.778093 27216 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d9013ef87e487684fce4bd2e081c3c" candidate_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45175aad53214b239a2976a1ff066dcf" is_pre_election: true
I20251210 21:06:00.778410 27141 raft_consensus.cc:2468] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d5cbf0a0edc4610a2115a8e6c5d2429 in term 0.
I20251210 21:06:00.778407 27216 raft_consensus.cc:2468] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d5cbf0a0edc4610a2115a8e6c5d2429 in term 0.
I20251210 21:06:00.778966 27255 leader_election.cc:304] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 45175aad53214b239a2976a1ff066dcf, 4d5cbf0a0edc4610a2115a8e6c5d2429, c25e6a66028540a599247b4aaddb4150; no voters:
I20251210 21:06:00.779277 27341 raft_consensus.cc:2804] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:00.779376 27341 raft_consensus.cc:493] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:00.779484 27341 raft_consensus.cc:3060] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:00.781505 27341 raft_consensus.cc:515] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.782083 27341 leader_election.cc:290] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [CANDIDATE]: Term 1 election: Requested vote from peers 45175aad53214b239a2976a1ff066dcf (127.21.127.66:43227), c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929)
I20251210 21:06:00.782542 27141 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d9013ef87e487684fce4bd2e081c3c" candidate_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25e6a66028540a599247b4aaddb4150"
I20251210 21:06:00.782703 27141 raft_consensus.cc:3060] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:00.783643 27216 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d9013ef87e487684fce4bd2e081c3c" candidate_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45175aad53214b239a2976a1ff066dcf"
I20251210 21:06:00.783826 27216 raft_consensus.cc:3060] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:00.785137 27141 raft_consensus.cc:2468] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d5cbf0a0edc4610a2115a8e6c5d2429 in term 1.
I20251210 21:06:00.785564 27255 leader_election.cc:304] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d5cbf0a0edc4610a2115a8e6c5d2429, c25e6a66028540a599247b4aaddb4150; no voters:
I20251210 21:06:00.785887 27341 raft_consensus.cc:2804] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:00.786203 27216 raft_consensus.cc:2468] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d5cbf0a0edc4610a2115a8e6c5d2429 in term 1.
I20251210 21:06:00.786343 27341 raft_consensus.cc:697] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 1 LEADER]: Becoming Leader. State: Replica: 4d5cbf0a0edc4610a2115a8e6c5d2429, State: Running, Role: LEADER
I20251210 21:06:00.786643 27341 consensus_queue.cc:237] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } }
I20251210 21:06:00.789502 27038 catalog_manager.cc:5654] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d5cbf0a0edc4610a2115a8e6c5d2429 (127.21.127.67). New cstate: current_term: 1 leader_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d5cbf0a0edc4610a2115a8e6c5d2429" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 35535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c25e6a66028540a599247b4aaddb4150" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 44929 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:00.800241 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:06:00.807420 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:00.807848 22013 tablet_replica.cc:333] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150: stopping tablet replica
I20251210 21:06:00.808049 22013 raft_consensus.cc:2243] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:00.808257 22013 raft_consensus.cc:2272] T 80d9013ef87e487684fce4bd2e081c3c P c25e6a66028540a599247b4aaddb4150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:00.822129 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:06:01.229560 27341 consensus_queue.cc:1048] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45175aad53214b239a2976a1ff066dcf" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 43227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:06:01.235064 27255 consensus_peers.cc:597] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 -> Peer c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929): Couldn't send request to peer c25e6a66028540a599247b4aaddb4150. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:44929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:01.494086 27093 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:06:01.494483 27093 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:06:01.494580 27093 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:06:03.495044 27093 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:06:03.495607 27093 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:06:03.495745 27093 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251210 21:06:03.768069 27255 consensus_peers.cc:597] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 -> Peer c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929): Couldn't send request to peer c25e6a66028540a599247b4aaddb4150. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:44929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251210 21:06:04.356717 27255 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.65:44929: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20251210 21:06:05.496173 27093 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:06:05.496636 27093 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:06:05.496737 27093 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251210 21:06:06.248211 27255 consensus_peers.cc:597] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 -> Peer c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929): Couldn't send request to peer c25e6a66028540a599247b4aaddb4150. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:44929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251210 21:06:07.497157 27093 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:06:07.497604 27093 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:06:07.497713 27093 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251210 21:06:08.903766 27255 consensus_peers.cc:597] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 -> Peer c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929): Couldn't send request to peer c25e6a66028540a599247b4aaddb4150. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:44929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251210 21:06:09.451876 27255 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.65:44929: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20251210 21:06:09.498123 27093 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:06:09.498580 27093 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:06:09.498683 27093 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251210 21:06:11.401739 27255 consensus_peers.cc:597] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 -> Peer c25e6a66028540a599247b4aaddb4150 (127.21.127.65:44929): Couldn't send request to peer c25e6a66028540a599247b4aaddb4150. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:44929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20251210 21:06:11.499153 27093 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251210 21:06:11.499717 27093 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251210 21:06:11.499830 27093 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251210 21:06:11.879513 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:06:11.921486 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:11.921753 22013 tablet_replica.cc:333] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf: stopping tablet replica
I20251210 21:06:11.921928 22013 raft_consensus.cc:2243] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:11.922129 22013 raft_consensus.cc:2272] T 80d9013ef87e487684fce4bd2e081c3c P 45175aad53214b239a2976a1ff066dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:11.936023 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:06:11.939132 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:06:11.946817 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:11.947094 22013 tablet_replica.cc:333] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429: stopping tablet replica
I20251210 21:06:11.947255 22013 raft_consensus.cc:2243] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:11.947607 22013 raft_consensus.cc:2272] T 80d9013ef87e487684fce4bd2e081c3c P 4d5cbf0a0edc4610a2115a8e6c5d2429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:11.961277 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:06:11.964294 22013 master.cc:561] Master@127.21.127.126:38205 shutting down...
I20251210 21:06:11.972883 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:11.973176 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:11.973281 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4be93477b3924c41a22e5c00906a576b: stopping tablet replica
I20251210 21:06:11.986713 22013 master.cc:583] Master@127.21.127.126:38205 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor (12561 ms)
[ RUN ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering
I20251210 21:06:11.997655 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:35949
I20251210 21:06:11.998207 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:12.000576 27374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:12.000659 27375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:12.000844 27377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:12.001142 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:12.001489 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:12.001545 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:12.001574 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400772001570 us; error 0 us; skew 500 ppm
I20251210 21:06:12.001741 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:12.002980 22013 webserver.cc:492] Webserver started at http://127.21.127.126:41937/ using document root <none> and password file <none>
I20251210 21:06:12.003201 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:12.003264 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:12.003540 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:12.004324 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "26c4f150a2844647be598b98116f6cf6"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.007176 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20251210 21:06:12.009084 27382 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.009467 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:12.009569 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "26c4f150a2844647be598b98116f6cf6"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.009686 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:12.018254 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:12.018784 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:12.030303 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:35949
I20251210 21:06:12.030328 27433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:35949 every 8 connection(s)
I20251210 21:06:12.031646 27434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:12.035702 27434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6: Bootstrap starting.
I20251210 21:06:12.036983 27434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:12.038475 27434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6: No bootstrap required, opened a new log
I20251210 21:06:12.039054 27434 raft_consensus.cc:359] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c4f150a2844647be598b98116f6cf6" member_type: VOTER }
I20251210 21:06:12.039171 27434 raft_consensus.cc:385] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:12.039203 27434 raft_consensus.cc:740] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c4f150a2844647be598b98116f6cf6, State: Initialized, Role: FOLLOWER
I20251210 21:06:12.039366 27434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c4f150a2844647be598b98116f6cf6" member_type: VOTER }
I20251210 21:06:12.039511 27434 raft_consensus.cc:399] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:06:12.039556 27434 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:06:12.039608 27434 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:12.041294 27434 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c4f150a2844647be598b98116f6cf6" member_type: VOTER }
I20251210 21:06:12.041458 27434 leader_election.cc:304] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c4f150a2844647be598b98116f6cf6; no voters:
I20251210 21:06:12.041785 27434 leader_election.cc:290] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:06:12.041900 27437 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:12.042322 27437 raft_consensus.cc:697] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 1 LEADER]: Becoming Leader. State: Replica: 26c4f150a2844647be598b98116f6cf6, State: Running, Role: LEADER
I20251210 21:06:12.042531 27434 sys_catalog.cc:565] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:06:12.042526 27437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c4f150a2844647be598b98116f6cf6" member_type: VOTER }
I20251210 21:06:12.043330 27439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c4f150a2844647be598b98116f6cf6. Latest consensus state: current_term: 1 leader_uuid: "26c4f150a2844647be598b98116f6cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c4f150a2844647be598b98116f6cf6" member_type: VOTER } }
I20251210 21:06:12.043385 27438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c4f150a2844647be598b98116f6cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c4f150a2844647be598b98116f6cf6" member_type: VOTER } }
I20251210 21:06:12.043484 27439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [sys.catalog]: This master's current role is: LEADER
I20251210 21:06:12.043534 27438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [sys.catalog]: This master's current role is: LEADER
I20251210 21:06:12.052275 27442 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:06:12.052551 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:06:12.053678 27442 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:06:12.056757 27442 catalog_manager.cc:1357] Generated new cluster ID: 80dd4a1f3cea4a1797c7f275871b3ebb
I20251210 21:06:12.056826 27442 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:06:12.073431 27442 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:06:12.074199 27442 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:06:12.083231 27442 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6: Generated new TSK 0
I20251210 21:06:12.083472 27442 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:06:12.085430 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:12.088179 27455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:12.088225 27456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:12.088418 27458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:12.088532 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:12.088984 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:12.089042 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:12.089071 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400772089068 us; error 0 us; skew 500 ppm
I20251210 21:06:12.089210 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:12.090317 22013 webserver.cc:492] Webserver started at http://127.21.127.65:42943/ using document root <none> and password file <none>
I20251210 21:06:12.090533 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:12.090595 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:12.090713 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:12.091374 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "555272d9a8304861a478617374fdd937"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.093865 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:12.095507 27463 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.095824 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:12.095918 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "555272d9a8304861a478617374fdd937"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.096028 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:12.105829 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:12.106395 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:12.106931 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:12.107878 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:12.107935 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.107999 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:12.108042 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.120110 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:33525
I20251210 21:06:12.120139 27525 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:33525 every 8 connection(s)
I20251210 21:06:12.121560 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:12.124593 27530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:12.124891 27531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:12.124891 27533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:12.125875 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:12.126148 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:12.126215 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:12.126271 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400772126245 us; error 0 us; skew 500 ppm
I20251210 21:06:12.126423 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:12.127779 22013 webserver.cc:492] Webserver started at http://127.21.127.66:34609/ using document root <none> and password file <none>
I20251210 21:06:12.128074 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:12.128178 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:12.128319 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:12.128682 27526 heartbeater.cc:344] Connected to a master server at 127.21.127.126:35949
I20251210 21:06:12.128829 27526 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:12.129089 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "f9faf037af0f42589e31251f6888af5a"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.129231 27526 heartbeater.cc:507] Master 127.21.127.126:35949 requested a full tablet report, sending...
I20251210 21:06:12.130141 27399 ts_manager.cc:194] Registered new tserver with Master: 555272d9a8304861a478617374fdd937 (127.21.127.65:33525)
I20251210 21:06:12.131686 27399 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60244
I20251210 21:06:12.132058 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251210 21:06:12.133558 27538 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.133868 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:06:12.134022 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "f9faf037af0f42589e31251f6888af5a"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.134136 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:12.168608 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:12.169239 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:12.169812 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:12.170728 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:12.170797 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.170883 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:12.170943 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.184712 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:39767
I20251210 21:06:12.184758 27600 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:39767 every 8 connection(s)
I20251210 21:06:12.186271 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:12.189256 27604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:12.189738 27605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:12.192468 22013 server_base.cc:1047] running on GCE node
W20251210 21:06:12.192507 27607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:12.192960 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:12.193065 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:12.193167 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400772193109 us; error 0 us; skew 500 ppm
I20251210 21:06:12.193161 27601 heartbeater.cc:344] Connected to a master server at 127.21.127.126:35949
I20251210 21:06:12.193365 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:12.193450 27601 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:12.193869 27601 heartbeater.cc:507] Master 127.21.127.126:35949 requested a full tablet report, sending...
I20251210 21:06:12.194659 22013 webserver.cc:492] Webserver started at http://127.21.127.67:42831/ using document root <none> and password file <none>
I20251210 21:06:12.194924 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:12.194926 27399 ts_manager.cc:194] Registered new tserver with Master: f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:12.195078 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:12.195227 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:12.195977 27399 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60260
I20251210 21:06:12.196039 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "6a48229cbada4c4fbabca282b5525be2"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.198562 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:12.200142 27612 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.200609 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:12.200714 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "6a48229cbada4c4fbabca282b5525be2"
format_stamp: "Formatted at 2025-12-10 21:06:12 on dist-test-slave-9pq8"
I20251210 21:06:12.200837 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:12.209303 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:12.209844 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:12.210418 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:12.211289 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:12.211371 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.211501 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:12.211553 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:12.222985 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:36493
I20251210 21:06:12.223011 27674 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:36493 every 8 connection(s)
I20251210 21:06:12.230160 27675 heartbeater.cc:344] Connected to a master server at 127.21.127.126:35949
I20251210 21:06:12.230335 27675 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:12.230726 27675 heartbeater.cc:507] Master 127.21.127.126:35949 requested a full tablet report, sending...
I20251210 21:06:12.231607 27399 ts_manager.cc:194] Registered new tserver with Master: 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:12.231825 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007966827s
I20251210 21:06:12.232455 27399 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60276
I20251210 21:06:13.133643 27526 heartbeater.cc:499] Master 127.21.127.126:35949 was elected leader, sending a full tablet report...
I20251210 21:06:13.197405 27601 heartbeater.cc:499] Master 127.21.127.126:35949 was elected leader, sending a full tablet report...
I20251210 21:06:13.233707 27675 heartbeater.cc:499] Master 127.21.127.126:35949 was elected leader, sending a full tablet report...
I20251210 21:06:13.259725 22013 test_util.cc:276] Using random seed: 1710879145
I20251210 21:06:13.276839 27399 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60288:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:06:13.277541 27399 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:06:13.306327 27491 tablet_service.cc:1505] Processing CreateTablet for tablet 669859794b47427888e0c01cb869bad5 (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:13.306568 27489 tablet_service.cc:1505] Processing CreateTablet for tablet 985fb17e8948485b895e8edcd21bde2c (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:13.306883 27491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 669859794b47427888e0c01cb869bad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.307297 27488 tablet_service.cc:1505] Processing CreateTablet for tablet 576d8355b30b4cd0bed786091ba63f1d (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:13.307714 27488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 576d8355b30b4cd0bed786091ba63f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.310716 27489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985fb17e8948485b895e8edcd21bde2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.306327 27490 tablet_service.cc:1505] Processing CreateTablet for tablet 571081fa9d9e4637b74c00b9a2035026 (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:13.313702 27490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571081fa9d9e4637b74c00b9a2035026. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.314139 27695 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: Bootstrap starting.
I20251210 21:06:13.315697 27640 tablet_service.cc:1505] Processing CreateTablet for tablet 669859794b47427888e0c01cb869bad5 (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:13.315775 27566 tablet_service.cc:1505] Processing CreateTablet for tablet 669859794b47427888e0c01cb869bad5 (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:13.316126 27640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 669859794b47427888e0c01cb869bad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.316156 27566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 669859794b47427888e0c01cb869bad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.320168 27639 tablet_service.cc:1505] Processing CreateTablet for tablet 571081fa9d9e4637b74c00b9a2035026 (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:13.320591 27639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571081fa9d9e4637b74c00b9a2035026. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.323051 27565 tablet_service.cc:1505] Processing CreateTablet for tablet 571081fa9d9e4637b74c00b9a2035026 (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:13.323136 27564 tablet_service.cc:1505] Processing CreateTablet for tablet 985fb17e8948485b895e8edcd21bde2c (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:13.323283 27696 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a: Bootstrap starting.
I20251210 21:06:13.323690 27563 tablet_service.cc:1505] Processing CreateTablet for tablet 576d8355b30b4cd0bed786091ba63f1d (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:13.324054 27565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571081fa9d9e4637b74c00b9a2035026. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.324285 27563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 576d8355b30b4cd0bed786091ba63f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.324359 27638 tablet_service.cc:1505] Processing CreateTablet for tablet 985fb17e8948485b895e8edcd21bde2c (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:13.324751 27638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985fb17e8948485b895e8edcd21bde2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.328387 27695 tablet_bootstrap.cc:654] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.331063 27637 tablet_service.cc:1505] Processing CreateTablet for tablet 576d8355b30b4cd0bed786091ba63f1d (DEFAULT_TABLE table=test-workload [id=fb5fc0a6d311451aa431342f45a02895]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:13.331533 27637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 576d8355b30b4cd0bed786091ba63f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.331597 27564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985fb17e8948485b895e8edcd21bde2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:13.331784 27695 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: No bootstrap required, opened a new log
I20251210 21:06:13.331928 27695 ts_tablet_manager.cc:1403] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20251210 21:06:13.332495 27696 tablet_bootstrap.cc:654] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.333782 27695 raft_consensus.cc:359] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.333964 27695 raft_consensus.cc:385] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.334033 27695 raft_consensus.cc:740] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555272d9a8304861a478617374fdd937, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.334286 27695 consensus_queue.cc:260] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.335336 27697 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: Bootstrap starting.
I20251210 21:06:13.337227 27697 tablet_bootstrap.cc:654] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.339013 27695 ts_tablet_manager.cc:1434] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20251210 21:06:13.339587 27696 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a: No bootstrap required, opened a new log
I20251210 21:06:13.339839 27696 ts_tablet_manager.cc:1403] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.008s
I20251210 21:06:13.339408 27695 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937: Bootstrap starting.
I20251210 21:06:13.340668 27696 raft_consensus.cc:359] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.340889 27696 raft_consensus.cc:385] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.340999 27696 raft_consensus.cc:740] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9faf037af0f42589e31251f6888af5a, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.341284 27696 consensus_queue.cc:260] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.341514 27695 tablet_bootstrap.cc:654] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.342847 27696 ts_tablet_manager.cc:1434] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251210 21:06:13.342854 27697 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: No bootstrap required, opened a new log
I20251210 21:06:13.343266 27696 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a: Bootstrap starting.
I20251210 21:06:13.343335 27697 ts_tablet_manager.cc:1403] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20251210 21:06:13.343676 27695 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937: No bootstrap required, opened a new log
I20251210 21:06:13.343809 27695 ts_tablet_manager.cc:1403] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:13.344487 27695 raft_consensus.cc:359] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.344748 27695 raft_consensus.cc:385] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.344888 27695 raft_consensus.cc:740] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555272d9a8304861a478617374fdd937, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.345099 27696 tablet_bootstrap.cc:654] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.345081 27697 raft_consensus.cc:359] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.345248 27695 consensus_queue.cc:260] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.345839 27695 ts_tablet_manager.cc:1434] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:13.346336 27695 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: Bootstrap starting.
I20251210 21:06:13.345383 27697 raft_consensus.cc:385] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.346845 27697 raft_consensus.cc:740] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a48229cbada4c4fbabca282b5525be2, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.347198 27697 consensus_queue.cc:260] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.348031 27695 tablet_bootstrap.cc:654] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.352118 27697 ts_tablet_manager.cc:1434] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: Time spent starting tablet: real 0.009s user 0.003s sys 0.002s
I20251210 21:06:13.352937 27696 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a: No bootstrap required, opened a new log
I20251210 21:06:13.353112 27696 ts_tablet_manager.cc:1403] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20251210 21:06:13.353916 27696 raft_consensus.cc:359] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.354115 27696 raft_consensus.cc:385] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.354209 27696 raft_consensus.cc:740] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9faf037af0f42589e31251f6888af5a, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.354491 27696 consensus_queue.cc:260] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.354964 27696 ts_tablet_manager.cc:1434] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:13.355260 27696 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: Bootstrap starting.
I20251210 21:06:13.357020 27696 tablet_bootstrap.cc:654] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.357111 27697 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2: Bootstrap starting.
I20251210 21:06:13.359246 27696 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: No bootstrap required, opened a new log
I20251210 21:06:13.359472 27696 ts_tablet_manager.cc:1403] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:13.359597 27697 tablet_bootstrap.cc:654] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.359927 27695 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: No bootstrap required, opened a new log
I20251210 21:06:13.360092 27695 ts_tablet_manager.cc:1403] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20251210 21:06:13.360258 27696 raft_consensus.cc:359] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.360419 27696 raft_consensus.cc:385] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.360508 27696 raft_consensus.cc:740] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9faf037af0f42589e31251f6888af5a, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.360735 27696 consensus_queue.cc:260] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.360838 27695 raft_consensus.cc:359] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.361071 27695 raft_consensus.cc:385] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.361176 27695 raft_consensus.cc:740] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555272d9a8304861a478617374fdd937, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.361222 27696 ts_tablet_manager.cc:1434] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:06:13.361414 27695 consensus_queue.cc:260] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.362629 27695 ts_tablet_manager.cc:1434] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:13.362982 27695 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: Bootstrap starting.
I20251210 21:06:13.364809 27695 tablet_bootstrap.cc:654] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.372521 27697 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2: No bootstrap required, opened a new log
I20251210 21:06:13.372655 27697 ts_tablet_manager.cc:1403] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20251210 21:06:13.372893 27695 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: No bootstrap required, opened a new log
I20251210 21:06:13.373025 27695 ts_tablet_manager.cc:1403] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20251210 21:06:13.373350 27697 raft_consensus.cc:359] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.373498 27697 raft_consensus.cc:385] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.373569 27697 raft_consensus.cc:740] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a48229cbada4c4fbabca282b5525be2, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.373636 27695 raft_consensus.cc:359] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.373788 27695 raft_consensus.cc:385] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.373768 27697 consensus_queue.cc:260] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.373878 27695 raft_consensus.cc:740] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555272d9a8304861a478617374fdd937, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.374094 27695 consensus_queue.cc:260] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.374236 27697 ts_tablet_manager.cc:1434] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:13.374506 27697 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: Bootstrap starting.
I20251210 21:06:13.375845 27695 ts_tablet_manager.cc:1434] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251210 21:06:13.376219 27697 tablet_bootstrap.cc:654] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.379029 27697 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: No bootstrap required, opened a new log
I20251210 21:06:13.379143 27697 ts_tablet_manager.cc:1403] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251210 21:06:13.379702 27697 raft_consensus.cc:359] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.379827 27697 raft_consensus.cc:385] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.379889 27697 raft_consensus.cc:740] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a48229cbada4c4fbabca282b5525be2, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.380055 27697 consensus_queue.cc:260] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.380411 27697 ts_tablet_manager.cc:1434] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:13.380671 27697 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: Bootstrap starting.
I20251210 21:06:13.381873 27697 tablet_bootstrap.cc:654] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.383329 27697 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: No bootstrap required, opened a new log
I20251210 21:06:13.383471 27697 ts_tablet_manager.cc:1403] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251210 21:06:13.384080 27697 raft_consensus.cc:359] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.384223 27697 raft_consensus.cc:385] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.384294 27697 raft_consensus.cc:740] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a48229cbada4c4fbabca282b5525be2, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.384398 27696 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a: Bootstrap starting.
I20251210 21:06:13.384466 27697 consensus_queue.cc:260] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.384903 27697 ts_tablet_manager.cc:1434] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:13.385875 27696 tablet_bootstrap.cc:654] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:13.387516 27696 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a: No bootstrap required, opened a new log
I20251210 21:06:13.387694 27696 ts_tablet_manager.cc:1403] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20251210 21:06:13.388264 27696 raft_consensus.cc:359] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.388382 27696 raft_consensus.cc:385] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:13.388437 27696 raft_consensus.cc:740] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9faf037af0f42589e31251f6888af5a, State: Initialized, Role: FOLLOWER
I20251210 21:06:13.388597 27696 consensus_queue.cc:260] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.388971 27696 ts_tablet_manager.cc:1434] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:13.453769 27700 raft_consensus.cc:493] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:13.453967 27700 raft_consensus.cc:515] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.454758 27700 leader_election.cc:290] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:13.461546 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "669859794b47427888e0c01cb869bad5" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2" is_pre_election: true
I20251210 21:06:13.461791 27650 raft_consensus.cc:2468] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 0.
I20251210 21:06:13.462185 27467 leader_election.cc:304] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555272d9a8304861a478617374fdd937, 6a48229cbada4c4fbabca282b5525be2; no voters:
I20251210 21:06:13.462311 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "669859794b47427888e0c01cb869bad5" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a" is_pre_election: true
I20251210 21:06:13.462435 27700 raft_consensus.cc:2804] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:13.462492 27576 raft_consensus.cc:2468] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 0.
I20251210 21:06:13.462510 27700 raft_consensus.cc:493] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:13.462594 27700 raft_consensus.cc:3060] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.464542 27700 raft_consensus.cc:515] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.465008 27700 leader_election.cc:290] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 election: Requested vote from peers 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:13.465394 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "669859794b47427888e0c01cb869bad5" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2"
I20251210 21:06:13.465444 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "669859794b47427888e0c01cb869bad5" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a"
I20251210 21:06:13.465562 27576 raft_consensus.cc:3060] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.465562 27650 raft_consensus.cc:3060] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.468197 27650 raft_consensus.cc:2468] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 1.
I20251210 21:06:13.468197 27576 raft_consensus.cc:2468] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 1.
I20251210 21:06:13.468639 27467 leader_election.cc:304] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555272d9a8304861a478617374fdd937, 6a48229cbada4c4fbabca282b5525be2; no voters:
I20251210 21:06:13.468909 27700 raft_consensus.cc:2804] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:13.469316 27700 raft_consensus.cc:697] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: Becoming Leader. State: Replica: 555272d9a8304861a478617374fdd937, State: Running, Role: LEADER
I20251210 21:06:13.469542 27700 consensus_queue.cc:237] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:13.471776 27399 catalog_manager.cc:5654] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 reported cstate change: term changed from 0 to 1, leader changed from <none> to 555272d9a8304861a478617374fdd937 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "555272d9a8304861a478617374fdd937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:13.484205 27700 raft_consensus.cc:493] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:13.484403 27700 raft_consensus.cc:515] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.484973 27700 leader_election.cc:290] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767), 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:13.485455 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "985fb17e8948485b895e8edcd21bde2c" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a" is_pre_election: true
I20251210 21:06:13.485505 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "985fb17e8948485b895e8edcd21bde2c" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2" is_pre_election: true
I20251210 21:06:13.485612 27576 raft_consensus.cc:2468] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 0.
I20251210 21:06:13.485683 27650 raft_consensus.cc:2468] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 0.
I20251210 21:06:13.485939 27467 leader_election.cc:304] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555272d9a8304861a478617374fdd937, f9faf037af0f42589e31251f6888af5a; no voters:
I20251210 21:06:13.486143 27700 raft_consensus.cc:2804] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:13.486218 27700 raft_consensus.cc:493] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:13.486276 27700 raft_consensus.cc:3060] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.488116 27700 raft_consensus.cc:515] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.488603 27700 leader_election.cc:290] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 election: Requested vote from peers f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767), 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:13.489025 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "985fb17e8948485b895e8edcd21bde2c" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2"
I20251210 21:06:13.489025 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "985fb17e8948485b895e8edcd21bde2c" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a"
I20251210 21:06:13.489213 27576 raft_consensus.cc:3060] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.489233 27650 raft_consensus.cc:3060] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.491744 27650 raft_consensus.cc:2468] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 1.
I20251210 21:06:13.491744 27576 raft_consensus.cc:2468] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 1.
I20251210 21:06:13.492111 27467 leader_election.cc:304] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555272d9a8304861a478617374fdd937, f9faf037af0f42589e31251f6888af5a; no voters:
I20251210 21:06:13.492321 27700 raft_consensus.cc:2804] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:13.492442 27700 raft_consensus.cc:697] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: Becoming Leader. State: Replica: 555272d9a8304861a478617374fdd937, State: Running, Role: LEADER
I20251210 21:06:13.492666 27700 consensus_queue.cc:237] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.495110 27399 catalog_manager.cc:5654] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 reported cstate change: term changed from 0 to 1, leader changed from <none> to 555272d9a8304861a478617374fdd937 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "555272d9a8304861a478617374fdd937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:13.533427 27700 raft_consensus.cc:493] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:13.533638 27700 raft_consensus.cc:515] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.534289 27700 leader_election.cc:290] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767), 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:13.534785 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "571081fa9d9e4637b74c00b9a2035026" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a" is_pre_election: true
I20251210 21:06:13.534785 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "571081fa9d9e4637b74c00b9a2035026" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2" is_pre_election: true
I20251210 21:06:13.535027 27650 raft_consensus.cc:2468] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 0.
I20251210 21:06:13.535027 27576 raft_consensus.cc:2468] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 0.
I20251210 21:06:13.535410 27467 leader_election.cc:304] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555272d9a8304861a478617374fdd937, 6a48229cbada4c4fbabca282b5525be2; no voters:
I20251210 21:06:13.535655 27700 raft_consensus.cc:2804] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:13.535737 27700 raft_consensus.cc:493] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:13.535822 27700 raft_consensus.cc:3060] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.537571 27700 raft_consensus.cc:515] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.538053 27700 leader_election.cc:290] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 election: Requested vote from peers f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767), 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:13.538452 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "571081fa9d9e4637b74c00b9a2035026" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a"
I20251210 21:06:13.538452 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "571081fa9d9e4637b74c00b9a2035026" candidate_uuid: "555272d9a8304861a478617374fdd937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2"
I20251210 21:06:13.538635 27576 raft_consensus.cc:3060] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.538647 27650 raft_consensus.cc:3060] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.541252 27650 raft_consensus.cc:2468] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 1.
I20251210 21:06:13.541252 27576 raft_consensus.cc:2468] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 555272d9a8304861a478617374fdd937 in term 1.
I20251210 21:06:13.541666 27467 leader_election.cc:304] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555272d9a8304861a478617374fdd937, 6a48229cbada4c4fbabca282b5525be2; no voters:
I20251210 21:06:13.541883 27700 raft_consensus.cc:2804] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:13.542006 27700 raft_consensus.cc:697] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: Becoming Leader. State: Replica: 555272d9a8304861a478617374fdd937, State: Running, Role: LEADER
I20251210 21:06:13.542446 27700 consensus_queue.cc:237] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.544785 27399 catalog_manager.cc:5654] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 reported cstate change: term changed from 0 to 1, leader changed from <none> to 555272d9a8304861a478617374fdd937 (127.21.127.65). New cstate: current_term: 1 leader_uuid: "555272d9a8304861a478617374fdd937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:13.553916 27705 raft_consensus.cc:493] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:13.554057 27705 raft_consensus.cc:515] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.554726 27705 leader_election.cc:290] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:13.560218 27501 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "576d8355b30b4cd0bed786091ba63f1d" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "555272d9a8304861a478617374fdd937" is_pre_election: true
I20251210 21:06:13.560411 27501 raft_consensus.cc:2468] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 0.
I20251210 21:06:13.560516 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "576d8355b30b4cd0bed786091ba63f1d" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a" is_pre_election: true
I20251210 21:06:13.560665 27576 raft_consensus.cc:2468] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 0.
I20251210 21:06:13.560849 27615 leader_election.cc:304] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555272d9a8304861a478617374fdd937, 6a48229cbada4c4fbabca282b5525be2; no voters:
I20251210 21:06:13.561091 27705 raft_consensus.cc:2804] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:13.561168 27705 raft_consensus.cc:493] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:13.561235 27705 raft_consensus.cc:3060] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.562995 27705 raft_consensus.cc:515] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.563531 27705 leader_election.cc:290] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 1 election: Requested vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:13.563879 27501 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "576d8355b30b4cd0bed786091ba63f1d" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "555272d9a8304861a478617374fdd937"
I20251210 21:06:13.563901 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "576d8355b30b4cd0bed786091ba63f1d" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a"
I20251210 21:06:13.564021 27501 raft_consensus.cc:3060] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.564041 27576 raft_consensus.cc:3060] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:13.565668 27576 raft_consensus.cc:2468] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 1.
I20251210 21:06:13.565692 27501 raft_consensus.cc:2468] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 1.
I20251210 21:06:13.565995 27616 leader_election.cc:304] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a48229cbada4c4fbabca282b5525be2, f9faf037af0f42589e31251f6888af5a; no voters:
I20251210 21:06:13.566241 27705 raft_consensus.cc:2804] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:13.566601 27705 raft_consensus.cc:697] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 LEADER]: Becoming Leader. State: Replica: 6a48229cbada4c4fbabca282b5525be2, State: Running, Role: LEADER
I20251210 21:06:13.566809 27705 consensus_queue.cc:237] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:13.569136 27399 catalog_manager.cc:5654] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67). New cstate: current_term: 1 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:13.600322 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:06:13.622435 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:13.622845 22013 tablet_replica.cc:333] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: stopping tablet replica
I20251210 21:06:13.623049 22013 raft_consensus.cc:2243] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:13.623319 22013 pending_rounds.cc:70] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: Trying to abort 1 pending ops.
I20251210 21:06:13.623368 22013 pending_rounds.cc:77] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081568227852288 op_type: NO_OP noop_request { }
I20251210 21:06:13.623510 22013 raft_consensus.cc:2889] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:13.623602 22013 raft_consensus.cc:2272] T 985fb17e8948485b895e8edcd21bde2c P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:13.624541 22013 tablet_replica.cc:333] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: stopping tablet replica
I20251210 21:06:13.624680 22013 raft_consensus.cc:2243] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:13.624859 22013 pending_rounds.cc:70] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: Trying to abort 1 pending ops.
I20251210 21:06:13.624898 22013 pending_rounds.cc:77] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081568432193536 op_type: NO_OP noop_request { }
I20251210 21:06:13.624987 22013 raft_consensus.cc:2889] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:13.625065 22013 raft_consensus.cc:2272] T 571081fa9d9e4637b74c00b9a2035026 P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:13.625756 22013 tablet_replica.cc:333] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: stopping tablet replica
I20251210 21:06:13.625875 22013 raft_consensus.cc:2243] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:13.626034 22013 pending_rounds.cc:70] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: Trying to abort 1 pending ops.
I20251210 21:06:13.626073 22013 pending_rounds.cc:77] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081568133210112 op_type: NO_OP noop_request { }
I20251210 21:06:13.626152 22013 raft_consensus.cc:2889] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:13.626233 22013 raft_consensus.cc:2272] T 669859794b47427888e0c01cb869bad5 P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:13.626870 22013 tablet_replica.cc:333] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937: stopping tablet replica
I20251210 21:06:13.626988 22013 raft_consensus.cc:2243] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:13.627100 22013 raft_consensus.cc:2272] T 576d8355b30b4cd0bed786091ba63f1d P 555272d9a8304861a478617374fdd937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:13.641662 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
W20251210 21:06:14.064287 27615 consensus_peers.cc:597] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:14.186292 27705 consensus_queue.cc:1048] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:14.579228 27705 consensus_queue.cc:579] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Leader has been unable to successfully communicate with peer 555272d9a8304861a478617374fdd937 for more than 1 seconds (1.012s)
W20251210 21:06:14.580523 27615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20251210 21:06:14.582254 27397 catalog_manager.cc:5332] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 576d8355b30b4cd0bed786091ba63f1d with cas_config_opid_index -1: no extra replica candidate found for tablet 576d8355b30b4cd0bed786091ba63f1d (table test-workload [id=fb5fc0a6d311451aa431342f45a02895]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20251210 21:06:14.645978 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:14.648830 27733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:14.649082 27736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:14.649199 27734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:14.649444 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:14.649704 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:14.649796 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:14.649843 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400774649840 us; error 0 us; skew 500 ppm
I20251210 21:06:14.649961 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:14.651084 22013 webserver.cc:492] Webserver started at http://127.21.127.68:35835/ using document root <none> and password file <none>
I20251210 21:06:14.651295 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:14.651346 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:14.651484 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:14.652097 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "39db45a3749e4ec8a3dc2a2af04d659e"
format_stamp: "Formatted at 2025-12-10 21:06:14 on dist-test-slave-9pq8"
I20251210 21:06:14.654728 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:14.656392 27741 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:14.656731 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251210 21:06:14.656822 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "39db45a3749e4ec8a3dc2a2af04d659e"
format_stamp: "Formatted at 2025-12-10 21:06:14 on dist-test-slave-9pq8"
I20251210 21:06:14.656946 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:14.685228 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:14.685834 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:14.686442 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:14.687378 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:14.687431 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:14.687546 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:14.687605 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:14.700001 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:39597
I20251210 21:06:14.700040 27803 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:39597 every 8 connection(s)
I20251210 21:06:14.706820 27804 heartbeater.cc:344] Connected to a master server at 127.21.127.126:35949
I20251210 21:06:14.706961 27804 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:14.707294 27804 heartbeater.cc:507] Master 127.21.127.126:35949 requested a full tablet report, sending...
I20251210 21:06:14.708101 27397 ts_manager.cc:194] Registered new tserver with Master: 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68:39597)
I20251210 21:06:14.708899 27397 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60298
I20251210 21:06:14.732338 27650 consensus_queue.cc:237] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:14.734642 27576 raft_consensus.cc:1275] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:06:14.735314 27727 consensus_queue.cc:1048] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251210 21:06:14.735880 27615 consensus_peers.cc:597] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:14.737491 27705 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:14.738183 27576 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:14.741353 27386 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 576d8355b30b4cd0bed786091ba63f1d with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 4)
W20251210 21:06:14.742461 27614 consensus_peers.cc:597] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 -> Peer 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68:39597): Couldn't send request to peer 39db45a3749e4ec8a3dc2a2af04d659e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 576d8355b30b4cd0bed786091ba63f1d. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:14.742889 27397 catalog_manager.cc:5654] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251210 21:06:15.053151 27447 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 555272d9a8304861a478617374fdd937 not available for placement
I20251210 21:06:15.100373 27705 raft_consensus.cc:493] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:15.100575 27705 raft_consensus.cc:515] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:15.101210 27705 leader_election.cc:290] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:15.101812 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "985fb17e8948485b895e8edcd21bde2c" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a" is_pre_election: true
I20251210 21:06:15.102030 27576 raft_consensus.cc:2468] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 1.
I20251210 21:06:15.102455 27616 leader_election.cc:304] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a48229cbada4c4fbabca282b5525be2, f9faf037af0f42589e31251f6888af5a; no voters:
W20251210 21:06:15.102632 27615 leader_election.cc:336] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:15.102731 27705 raft_consensus.cc:2804] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:06:15.102824 27705 raft_consensus.cc:493] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:15.102919 27705 raft_consensus.cc:3060] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:15.104802 27705 raft_consensus.cc:515] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:15.105338 27705 leader_election.cc:290] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 election: Requested vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:15.105553 27705 raft_consensus.cc:493] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:15.105687 27705 raft_consensus.cc:515] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:15.105839 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "985fb17e8948485b895e8edcd21bde2c" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a"
I20251210 21:06:15.106038 27576 raft_consensus.cc:3060] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:15.106386 27705 leader_election.cc:290] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
W20251210 21:06:15.106673 27615 leader_election.cc:336] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:15.106824 27575 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "669859794b47427888e0c01cb869bad5" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a" is_pre_election: true
I20251210 21:06:15.107033 27575 raft_consensus.cc:2468] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 1.
W20251210 21:06:15.107301 27615 leader_election.cc:336] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:15.107477 27616 leader_election.cc:304] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a48229cbada4c4fbabca282b5525be2, f9faf037af0f42589e31251f6888af5a; no voters: 555272d9a8304861a478617374fdd937
I20251210 21:06:15.107719 27705 raft_consensus.cc:2804] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:06:15.107807 27705 raft_consensus.cc:493] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:15.107895 27705 raft_consensus.cc:3060] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:15.108537 27576 raft_consensus.cc:2468] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 2.
I20251210 21:06:15.108897 27616 leader_election.cc:304] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a48229cbada4c4fbabca282b5525be2, f9faf037af0f42589e31251f6888af5a; no voters: 555272d9a8304861a478617374fdd937
I20251210 21:06:15.109180 27727 raft_consensus.cc:2804] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:06:15.109328 27727 raft_consensus.cc:697] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Becoming Leader. State: Replica: 6a48229cbada4c4fbabca282b5525be2, State: Running, Role: LEADER
I20251210 21:06:15.109563 27727 consensus_queue.cc:237] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:15.110085 27705 raft_consensus.cc:515] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:15.110651 27705 leader_election.cc:290] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 election: Requested vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:15.111059 27576 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "669859794b47427888e0c01cb869bad5" candidate_uuid: "6a48229cbada4c4fbabca282b5525be2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9faf037af0f42589e31251f6888af5a"
I20251210 21:06:15.111246 27576 raft_consensus.cc:3060] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:15.112012 27399 catalog_manager.cc:5654] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: term changed from 1 to 2, leader changed from 555272d9a8304861a478617374fdd937 (127.21.127.65) to 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67). New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } }
W20251210 21:06:15.112483 27615 leader_election.cc:336] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:15.113498 27576 raft_consensus.cc:2468] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a48229cbada4c4fbabca282b5525be2 in term 2.
I20251210 21:06:15.113852 27616 leader_election.cc:304] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a48229cbada4c4fbabca282b5525be2, f9faf037af0f42589e31251f6888af5a; no voters: 555272d9a8304861a478617374fdd937
I20251210 21:06:15.114200 27705 raft_consensus.cc:2804] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:06:15.114339 27705 raft_consensus.cc:697] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Becoming Leader. State: Replica: 6a48229cbada4c4fbabca282b5525be2, State: Running, Role: LEADER
I20251210 21:06:15.114713 27705 consensus_queue.cc:237] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } }
I20251210 21:06:15.116896 27399 catalog_manager.cc:5654] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: term changed from 1 to 2, leader changed from 555272d9a8304861a478617374fdd937 (127.21.127.65) to 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67). New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:15.118387 27817 ts_tablet_manager.cc:933] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: Initiating tablet copy from peer 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:15.118985 27817 tablet_copy_client.cc:323] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:36493
I20251210 21:06:15.124280 27660 tablet_copy_service.cc:140] P 6a48229cbada4c4fbabca282b5525be2: Received BeginTabletCopySession request for tablet 576d8355b30b4cd0bed786091ba63f1d from peer 39db45a3749e4ec8a3dc2a2af04d659e ({username='slave'} at 127.0.0.1:51130)
I20251210 21:06:15.124426 27660 tablet_copy_service.cc:161] P 6a48229cbada4c4fbabca282b5525be2: Beginning new tablet copy session on tablet 576d8355b30b4cd0bed786091ba63f1d from peer 39db45a3749e4ec8a3dc2a2af04d659e at {username='slave'} at 127.0.0.1:51130: session id = 39db45a3749e4ec8a3dc2a2af04d659e-576d8355b30b4cd0bed786091ba63f1d
I20251210 21:06:15.127265 27660 tablet_copy_source_session.cc:215] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:15.128547 27817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 576d8355b30b4cd0bed786091ba63f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:15.134495 27817 tablet_copy_client.cc:806] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:15.134765 27817 tablet_copy_client.cc:670] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:15.136833 27817 tablet_copy_client.cc:538] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:15.139932 27817 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap starting.
I20251210 21:06:15.144824 27817 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:15.145165 27817 tablet_bootstrap.cc:492] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap complete.
I20251210 21:06:15.145401 27817 ts_tablet_manager.cc:1403] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251210 21:06:15.146055 27817 raft_consensus.cc:359] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:15.146220 27817 raft_consensus.cc:740] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39db45a3749e4ec8a3dc2a2af04d659e, State: Initialized, Role: LEARNER
I20251210 21:06:15.146395 27817 consensus_queue.cc:260] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:15.147081 27817 ts_tablet_manager.cc:1434] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:15.147089 27804 heartbeater.cc:499] Master 127.21.127.126:35949 was elected leader, sending a full tablet report...
I20251210 21:06:15.147853 27660 tablet_copy_service.cc:342] P 6a48229cbada4c4fbabca282b5525be2: Request end of tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-576d8355b30b4cd0bed786091ba63f1d received from {username='slave'} at 127.0.0.1:51130
I20251210 21:06:15.147981 27660 tablet_copy_service.cc:434] P 6a48229cbada4c4fbabca282b5525be2: ending tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-576d8355b30b4cd0bed786091ba63f1d on tablet 576d8355b30b4cd0bed786091ba63f1d with peer 39db45a3749e4ec8a3dc2a2af04d659e
I20251210 21:06:15.195700 27810 raft_consensus.cc:493] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:15.195931 27810 raft_consensus.cc:515] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:15.196771 27810 leader_election.cc:290] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
W20251210 21:06:15.198917 27541 leader_election.cc:336] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:15.202692 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "571081fa9d9e4637b74c00b9a2035026" candidate_uuid: "f9faf037af0f42589e31251f6888af5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2" is_pre_election: true
I20251210 21:06:15.202898 27650 raft_consensus.cc:2468] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9faf037af0f42589e31251f6888af5a in term 1.
I20251210 21:06:15.203280 27542 leader_election.cc:304] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a48229cbada4c4fbabca282b5525be2, f9faf037af0f42589e31251f6888af5a; no voters: 555272d9a8304861a478617374fdd937
I20251210 21:06:15.203585 27810 raft_consensus.cc:2804] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:06:15.203691 27810 raft_consensus.cc:493] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:15.203763 27810 raft_consensus.cc:3060] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:15.205551 27810 raft_consensus.cc:515] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:15.206115 27810 leader_election.cc:290] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [CANDIDATE]: Term 2 election: Requested vote from peers 555272d9a8304861a478617374fdd937 (127.21.127.65:33525), 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:15.206629 27650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "571081fa9d9e4637b74c00b9a2035026" candidate_uuid: "f9faf037af0f42589e31251f6888af5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a48229cbada4c4fbabca282b5525be2"
I20251210 21:06:15.206828 27650 raft_consensus.cc:3060] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Advancing to term 2
W20251210 21:06:15.207501 27541 leader_election.cc:336] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:15.208997 27650 raft_consensus.cc:2468] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9faf037af0f42589e31251f6888af5a in term 2.
I20251210 21:06:15.209367 27542 leader_election.cc:304] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a48229cbada4c4fbabca282b5525be2, f9faf037af0f42589e31251f6888af5a; no voters: 555272d9a8304861a478617374fdd937
I20251210 21:06:15.209661 27810 raft_consensus.cc:2804] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:06:15.210023 27810 raft_consensus.cc:697] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 LEADER]: Becoming Leader. State: Replica: f9faf037af0f42589e31251f6888af5a, State: Running, Role: LEADER
I20251210 21:06:15.210273 27810 consensus_queue.cc:237] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } }
I20251210 21:06:15.212734 27399 catalog_manager.cc:5654] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a reported cstate change: term changed from 1 to 2, leader changed from 555272d9a8304861a478617374fdd937 (127.21.127.65) to f9faf037af0f42589e31251f6888af5a (127.21.127.66). New cstate: current_term: 2 leader_uuid: "f9faf037af0f42589e31251f6888af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:15.526609 27576 raft_consensus.cc:1275] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251210 21:06:15.527226 27705 consensus_queue.cc:1048] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:06:15.527820 27615 consensus_peers.cc:597] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251210 21:06:15.656767 27615 consensus_peers.cc:597] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:15.659538 27576 raft_consensus.cc:1275] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251210 21:06:15.660010 27705 consensus_queue.cc:1048] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:15.691316 27779 raft_consensus.cc:1217] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251210 21:06:15.705701 27650 raft_consensus.cc:1275] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Refusing update from remote peer f9faf037af0f42589e31251f6888af5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251210 21:06:15.706313 27810 consensus_queue.cc:1048] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:06:15.712868 27541 consensus_peers.cc:597] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:15.943634 27705 consensus_queue.cc:579] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Leader has been unable to successfully communicate with peer 555272d9a8304861a478617374fdd937 for more than 1 seconds (1.211s)
W20251210 21:06:16.053982 27447 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 555272d9a8304861a478617374fdd937 not available for placement
I20251210 21:06:16.080032 27727 raft_consensus.cc:1064] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: attempting to promote NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e to VOTER
I20251210 21:06:16.080750 27727 consensus_queue.cc:237] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:16.082715 27779 raft_consensus.cc:1275] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 LEARNER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:16.082738 27576 raft_consensus.cc:1275] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:16.083202 27727 consensus_queue.cc:1048] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:16.083429 27705 consensus_queue.cc:1048] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251210 21:06:16.083696 27615 consensus_peers.cc:597] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:16.085693 27727 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:16.086488 27779 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:16.089452 27576 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:16.090201 27399 catalog_manager.cc:5654] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:16.102667 27650 consensus_queue.cc:237] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:16.104487 27779 raft_consensus.cc:1275] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:06:16.104489 27576 raft_consensus.cc:1275] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:06:16.104959 27730 consensus_queue.cc:1048] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:16.105167 27705 consensus_queue.cc:1048] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:16.107270 27836 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:16.107801 27576 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:16.110237 27386 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 576d8355b30b4cd0bed786091ba63f1d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:16.111748 27398 catalog_manager.cc:5654] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a reported cstate change: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New cstate: current_term: 1 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:16.111748 27779 raft_consensus.cc:2955] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
W20251210 21:06:16.116734 27385 catalog_manager.cc:4712] TS 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 576d8355b30b4cd0bed786091ba63f1d: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:16.249811 27836 consensus_queue.cc:579] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Leader has been unable to successfully communicate with peer 555272d9a8304861a478617374fdd937 for more than 1 seconds (1.135s)
I20251210 21:06:16.253305 27650 consensus_queue.cc:237] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:16.255034 27576 raft_consensus.cc:1275] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20251210 21:06:16.255187 27614 consensus_peers.cc:597] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 -> Peer 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68:39597): Couldn't send request to peer 39db45a3749e4ec8a3dc2a2af04d659e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 669859794b47427888e0c01cb869bad5. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:16.255537 27836 consensus_queue.cc:1048] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251210 21:06:16.255810 27615 consensus_peers.cc:597] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:16.257210 27730 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:16.257771 27576 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:16.260088 27386 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 669859794b47427888e0c01cb869bad5 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251210 21:06:16.261353 27398 catalog_manager.cc:5654] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:16.409776 27836 consensus_queue.cc:579] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Leader has been unable to successfully communicate with peer 555272d9a8304861a478617374fdd937 for more than 1 seconds (1.300s)
I20251210 21:06:16.413339 27650 consensus_queue.cc:237] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:16.415033 27576 raft_consensus.cc:1275] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20251210 21:06:16.415246 27614 consensus_peers.cc:597] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 -> Peer 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68:39597): Couldn't send request to peer 39db45a3749e4ec8a3dc2a2af04d659e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 985fb17e8948485b895e8edcd21bde2c. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:16.415647 27836 consensus_queue.cc:1048] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251210 21:06:16.415851 27615 consensus_peers.cc:597] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:16.417440 27705 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:16.418059 27576 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:16.421264 27386 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 985fb17e8948485b895e8edcd21bde2c with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251210 21:06:16.422660 27397 catalog_manager.cc:5654] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:16.526440 27833 consensus_queue.cc:579] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Leader has been unable to successfully communicate with peer 555272d9a8304861a478617374fdd937 for more than 1 seconds (1.316s)
I20251210 21:06:16.530375 27576 consensus_queue.cc:237] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:16.532655 27650 raft_consensus.cc:1275] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Refusing update from remote peer f9faf037af0f42589e31251f6888af5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:06:16.533386 27845 consensus_queue.cc:1048] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251210 21:06:16.533764 27541 consensus_peers.cc:597] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:16.535589 27833 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:16.536266 27650 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
W20251210 21:06:16.539597 27540 consensus_peers.cc:597] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a -> Peer 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68:39597): Couldn't send request to peer 39db45a3749e4ec8a3dc2a2af04d659e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 571081fa9d9e4637b74c00b9a2035026. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:16.539753 27386 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 571081fa9d9e4637b74c00b9a2035026 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251210 21:06:16.541249 27397 catalog_manager.cc:5654] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: config changed from index -1 to 2, NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "f9faf037af0f42589e31251f6888af5a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } } }
I20251210 21:06:16.844604 27849 ts_tablet_manager.cc:933] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: Initiating tablet copy from peer 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:16.845136 27849 tablet_copy_client.cc:323] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:36493
I20251210 21:06:16.845866 27660 tablet_copy_service.cc:140] P 6a48229cbada4c4fbabca282b5525be2: Received BeginTabletCopySession request for tablet 669859794b47427888e0c01cb869bad5 from peer 39db45a3749e4ec8a3dc2a2af04d659e ({username='slave'} at 127.0.0.1:51130)
I20251210 21:06:16.845989 27660 tablet_copy_service.cc:161] P 6a48229cbada4c4fbabca282b5525be2: Beginning new tablet copy session on tablet 669859794b47427888e0c01cb869bad5 from peer 39db45a3749e4ec8a3dc2a2af04d659e at {username='slave'} at 127.0.0.1:51130: session id = 39db45a3749e4ec8a3dc2a2af04d659e-669859794b47427888e0c01cb869bad5
I20251210 21:06:16.847779 27660 tablet_copy_source_session.cc:215] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:16.848829 27849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 669859794b47427888e0c01cb869bad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:16.852882 27849 tablet_copy_client.cc:806] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:16.853055 27849 tablet_copy_client.cc:670] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:16.854460 27849 tablet_copy_client.cc:538] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:16.856665 27849 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap starting.
I20251210 21:06:16.861004 27849 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:16.861327 27849 tablet_bootstrap.cc:492] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap complete.
I20251210 21:06:16.861507 27849 ts_tablet_manager.cc:1403] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251210 21:06:16.862043 27849 raft_consensus.cc:359] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:16.862188 27849 raft_consensus.cc:740] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 39db45a3749e4ec8a3dc2a2af04d659e, State: Initialized, Role: LEARNER
I20251210 21:06:16.862324 27849 consensus_queue.cc:260] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:16.862886 27849 ts_tablet_manager.cc:1434] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251210 21:06:16.863632 27660 tablet_copy_service.cc:342] P 6a48229cbada4c4fbabca282b5525be2: Request end of tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-669859794b47427888e0c01cb869bad5 received from {username='slave'} at 127.0.0.1:51130
I20251210 21:06:16.863734 27660 tablet_copy_service.cc:434] P 6a48229cbada4c4fbabca282b5525be2: ending tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-669859794b47427888e0c01cb869bad5 on tablet 669859794b47427888e0c01cb869bad5 with peer 39db45a3749e4ec8a3dc2a2af04d659e
I20251210 21:06:16.959700 27849 ts_tablet_manager.cc:933] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: Initiating tablet copy from peer f9faf037af0f42589e31251f6888af5a (127.21.127.66:39767)
I20251210 21:06:16.960274 27849 tablet_copy_client.cc:323] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:39767
I20251210 21:06:16.965739 27586 tablet_copy_service.cc:140] P f9faf037af0f42589e31251f6888af5a: Received BeginTabletCopySession request for tablet 571081fa9d9e4637b74c00b9a2035026 from peer 39db45a3749e4ec8a3dc2a2af04d659e ({username='slave'} at 127.0.0.1:60468)
I20251210 21:06:16.965884 27586 tablet_copy_service.cc:161] P f9faf037af0f42589e31251f6888af5a: Beginning new tablet copy session on tablet 571081fa9d9e4637b74c00b9a2035026 from peer 39db45a3749e4ec8a3dc2a2af04d659e at {username='slave'} at 127.0.0.1:60468: session id = 39db45a3749e4ec8a3dc2a2af04d659e-571081fa9d9e4637b74c00b9a2035026
I20251210 21:06:16.967707 27586 tablet_copy_source_session.cc:215] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:16.968751 27849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571081fa9d9e4637b74c00b9a2035026. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:16.972828 27849 tablet_copy_client.cc:806] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:16.973023 27849 tablet_copy_client.cc:670] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:16.974377 27849 tablet_copy_client.cc:538] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:16.976653 27849 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap starting.
I20251210 21:06:16.981122 27849 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:16.981405 27849 tablet_bootstrap.cc:492] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap complete.
I20251210 21:06:16.981600 27849 ts_tablet_manager.cc:1403] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251210 21:06:16.982126 27849 raft_consensus.cc:359] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:16.982278 27849 raft_consensus.cc:740] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 39db45a3749e4ec8a3dc2a2af04d659e, State: Initialized, Role: LEARNER
I20251210 21:06:16.982435 27849 consensus_queue.cc:260] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:16.982829 27849 ts_tablet_manager.cc:1434] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:16.983544 27586 tablet_copy_service.cc:342] P f9faf037af0f42589e31251f6888af5a: Request end of tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-571081fa9d9e4637b74c00b9a2035026 received from {username='slave'} at 127.0.0.1:60468
I20251210 21:06:16.983659 27586 tablet_copy_service.cc:434] P f9faf037af0f42589e31251f6888af5a: ending tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-571081fa9d9e4637b74c00b9a2035026 on tablet 571081fa9d9e4637b74c00b9a2035026 with peer 39db45a3749e4ec8a3dc2a2af04d659e
I20251210 21:06:17.002698 27849 ts_tablet_manager.cc:933] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: Initiating tablet copy from peer 6a48229cbada4c4fbabca282b5525be2 (127.21.127.67:36493)
I20251210 21:06:17.003281 27849 tablet_copy_client.cc:323] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:36493
I20251210 21:06:17.003983 27660 tablet_copy_service.cc:140] P 6a48229cbada4c4fbabca282b5525be2: Received BeginTabletCopySession request for tablet 985fb17e8948485b895e8edcd21bde2c from peer 39db45a3749e4ec8a3dc2a2af04d659e ({username='slave'} at 127.0.0.1:51130)
I20251210 21:06:17.004122 27660 tablet_copy_service.cc:161] P 6a48229cbada4c4fbabca282b5525be2: Beginning new tablet copy session on tablet 985fb17e8948485b895e8edcd21bde2c from peer 39db45a3749e4ec8a3dc2a2af04d659e at {username='slave'} at 127.0.0.1:51130: session id = 39db45a3749e4ec8a3dc2a2af04d659e-985fb17e8948485b895e8edcd21bde2c
I20251210 21:06:17.006085 27660 tablet_copy_source_session.cc:215] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:17.007184 27849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985fb17e8948485b895e8edcd21bde2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:17.011693 27849 tablet_copy_client.cc:806] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:17.011884 27849 tablet_copy_client.cc:670] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:17.013187 27849 tablet_copy_client.cc:538] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:17.015548 27849 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap starting.
I20251210 21:06:17.019855 27849 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:17.020186 27849 tablet_bootstrap.cc:492] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: Bootstrap complete.
I20251210 21:06:17.020392 27849 ts_tablet_manager.cc:1403] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent bootstrapping tablet: real 0.005s user 0.006s sys 0.000s
I20251210 21:06:17.020920 27849 raft_consensus.cc:359] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:17.021083 27849 raft_consensus.cc:740] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 39db45a3749e4ec8a3dc2a2af04d659e, State: Initialized, Role: LEARNER
I20251210 21:06:17.021248 27849 consensus_queue.cc:260] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: true } }
I20251210 21:06:17.021656 27849 ts_tablet_manager.cc:1434] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:17.022374 27660 tablet_copy_service.cc:342] P 6a48229cbada4c4fbabca282b5525be2: Request end of tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-985fb17e8948485b895e8edcd21bde2c received from {username='slave'} at 127.0.0.1:51130
I20251210 21:06:17.022490 27660 tablet_copy_service.cc:434] P 6a48229cbada4c4fbabca282b5525be2: ending tablet copy session 39db45a3749e4ec8a3dc2a2af04d659e-985fb17e8948485b895e8edcd21bde2c on tablet 985fb17e8948485b895e8edcd21bde2c with peer 39db45a3749e4ec8a3dc2a2af04d659e
W20251210 21:06:17.054790 27447 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 555272d9a8304861a478617374fdd937 not available for placement
I20251210 21:06:17.318789 27779 raft_consensus.cc:1217] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
W20251210 21:06:17.320245 27385 catalog_manager.cc:4712] TS 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 576d8355b30b4cd0bed786091ba63f1d: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:17.552534 27779 raft_consensus.cc:1217] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20251210 21:06:17.618620 27779 raft_consensus.cc:1217] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20251210 21:06:17.619318 27866 consensus_queue.cc:579] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Leader has been unable to successfully communicate with peer 555272d9a8304861a478617374fdd937 for more than 1 seconds (1.089s)
I20251210 21:06:17.654965 27863 consensus_queue.cc:579] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Leader has been unable to successfully communicate with peer 555272d9a8304861a478617374fdd937 for more than 1 seconds (1.241s)
I20251210 21:06:17.703912 27863 raft_consensus.cc:1064] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: attempting to promote NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e to VOTER
I20251210 21:06:17.704594 27863 consensus_queue.cc:237] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:17.706802 27575 raft_consensus.cc:1275] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:06:17.706898 27779 raft_consensus.cc:1275] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:06:17.707356 27868 consensus_queue.cc:1048] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:17.707634 27863 consensus_queue.cc:1048] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251210 21:06:17.707808 27615 consensus_peers.cc:597] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:17.709903 27863 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.710474 27779 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.714068 27575 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.715050 27397 catalog_manager.cc:5654] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:17.720085 27650 consensus_queue.cc:237] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:17.721585 27779 raft_consensus.cc:1275] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:06:17.721585 27575 raft_consensus.cc:1275] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:06:17.722074 27863 consensus_queue.cc:1048] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:17.722654 27863 consensus_queue.cc:1048] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:17.723901 27861 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.724515 27779 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.725207 27575 raft_consensus.cc:2955] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.726621 27386 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 669859794b47427888e0c01cb869bad5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:17.729202 27399 catalog_manager.cc:5654] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 reported cstate change: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:17.982055 27866 raft_consensus.cc:1064] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: attempting to promote NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e to VOTER
I20251210 21:06:17.982779 27866 consensus_queue.cc:237] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:17.984982 27650 raft_consensus.cc:1275] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Refusing update from remote peer f9faf037af0f42589e31251f6888af5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:06:17.984984 27779 raft_consensus.cc:1275] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Refusing update from remote peer f9faf037af0f42589e31251f6888af5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:06:17.985550 27866 consensus_queue.cc:1048] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:17.985795 27876 consensus_queue.cc:1048] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251210 21:06:17.986114 27541 consensus_peers.cc:597] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:17.988183 27876 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.988804 27650 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.992566 27779 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:17.993032 27397 catalog_manager.cc:5654] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a reported cstate change: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "f9faf037af0f42589e31251f6888af5a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:17.997145 27575 consensus_queue.cc:237] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:17.999711 27650 raft_consensus.cc:1275] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Refusing update from remote peer f9faf037af0f42589e31251f6888af5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:06:18.000108 27779 raft_consensus.cc:1275] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Refusing update from remote peer f9faf037af0f42589e31251f6888af5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:06:18.000161 27833 consensus_queue.cc:1048] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:18.000550 27865 consensus_queue.cc:1048] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:18.002180 27833 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.002826 27650 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.003193 27779 raft_consensus.cc:2955] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.005023 27386 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 571081fa9d9e4637b74c00b9a2035026 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:18.006142 27399 catalog_manager.cc:5654] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a reported cstate change: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New cstate: current_term: 2 leader_uuid: "f9faf037af0f42589e31251f6888af5a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:18.048341 27868 raft_consensus.cc:1064] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: attempting to promote NON_VOTER 39db45a3749e4ec8a3dc2a2af04d659e to VOTER
I20251210 21:06:18.049083 27868 consensus_queue.cc:237] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:18.051155 27575 raft_consensus.cc:1275] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:06:18.051162 27779 raft_consensus.cc:1275] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 LEARNER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251210 21:06:18.051700 27705 consensus_queue.cc:1048] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251210 21:06:18.052016 27615 consensus_peers.cc:597] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 -> Peer 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): Couldn't send request to peer 555272d9a8304861a478617374fdd937. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:18.052026 27868 consensus_queue.cc:1048] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:18.054523 27705 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.055389 27575 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
W20251210 21:06:18.056151 27447 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 555272d9a8304861a478617374fdd937 not available for placement
I20251210 21:06:18.058312 27779 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.059221 27398 catalog_manager.cc:5654] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a reported cstate change: config changed from index 2 to 3, 39db45a3749e4ec8a3dc2a2af04d659e (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "555272d9a8304861a478617374fdd937" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 33525 } } peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.063608 27650 consensus_queue.cc:237] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } }
I20251210 21:06:18.065171 27779 raft_consensus.cc:1275] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:06:18.065266 27575 raft_consensus.cc:1275] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Refusing update from remote peer 6a48229cbada4c4fbabca282b5525be2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251210 21:06:18.065647 27705 consensus_queue.cc:1048] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:18.066314 27705 consensus_queue.cc:1048] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:18.067589 27868 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.068133 27779 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.070379 27386 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 985fb17e8948485b895e8edcd21bde2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:18.071342 27575 raft_consensus.cc:2955] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
I20251210 21:06:18.071727 27399 catalog_manager.cc:5654] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e reported cstate change: config changed from index 3 to 4, VOTER 555272d9a8304861a478617374fdd937 (127.21.127.65) evicted. New cstate: current_term: 2 leader_uuid: "6a48229cbada4c4fbabca282b5525be2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f9faf037af0f42589e31251f6888af5a" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 39767 } } peers { permanent_uuid: "6a48229cbada4c4fbabca282b5525be2" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36493 } } peers { permanent_uuid: "39db45a3749e4ec8a3dc2a2af04d659e" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39597 } attrs { promote: false } } }
W20251210 21:06:18.332105 27385 catalog_manager.cc:4712] TS 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 571081fa9d9e4637b74c00b9a2035026: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
W20251210 21:06:19.963091 27385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111) [suppressed 61 similar messages]
W20251210 21:06:19.964275 27385 catalog_manager.cc:4712] TS 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 669859794b47427888e0c01cb869bad5: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:21.728272 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:06:21.736785 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:21.737119 22013 tablet_replica.cc:333] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a: stopping tablet replica
I20251210 21:06:21.737329 22013 raft_consensus.cc:2243] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.737592 22013 raft_consensus.cc:2272] T 985fb17e8948485b895e8edcd21bde2c P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.738247 22013 tablet_replica.cc:333] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a: stopping tablet replica
I20251210 21:06:21.738389 22013 raft_consensus.cc:2243] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:06:21.738688 22013 raft_consensus.cc:2272] T 571081fa9d9e4637b74c00b9a2035026 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.739189 22013 tablet_replica.cc:333] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a: stopping tablet replica
I20251210 21:06:21.739327 22013 raft_consensus.cc:2243] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.739575 22013 raft_consensus.cc:2272] T 669859794b47427888e0c01cb869bad5 P f9faf037af0f42589e31251f6888af5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.740005 22013 tablet_replica.cc:333] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a: stopping tablet replica
I20251210 21:06:21.740114 22013 raft_consensus.cc:2243] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.740312 22013 raft_consensus.cc:2272] T 576d8355b30b4cd0bed786091ba63f1d P f9faf037af0f42589e31251f6888af5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.754765 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:06:21.758740 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:06:21.766845 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:21.767189 22013 tablet_replica.cc:333] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2: stopping tablet replica
I20251210 21:06:21.767364 22013 raft_consensus.cc:2243] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:06:21.767751 22013 raft_consensus.cc:2272] T 985fb17e8948485b895e8edcd21bde2c P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.768379 22013 tablet_replica.cc:333] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2: stopping tablet replica
I20251210 21:06:21.768534 22013 raft_consensus.cc:2243] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:21.768813 22013 raft_consensus.cc:2272] T 576d8355b30b4cd0bed786091ba63f1d P 6a48229cbada4c4fbabca282b5525be2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.769353 22013 tablet_replica.cc:333] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2: stopping tablet replica
I20251210 21:06:21.769488 22013 raft_consensus.cc:2243] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:06:21.769731 22013 raft_consensus.cc:2272] T 669859794b47427888e0c01cb869bad5 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.770177 22013 tablet_replica.cc:333] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2: stopping tablet replica
I20251210 21:06:21.770304 22013 raft_consensus.cc:2243] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.770517 22013 raft_consensus.cc:2272] T 571081fa9d9e4637b74c00b9a2035026 P 6a48229cbada4c4fbabca282b5525be2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.784835 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:06:21.801759 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:06:21.810701 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:21.810961 22013 tablet_replica.cc:333] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e: stopping tablet replica
I20251210 21:06:21.811151 22013 raft_consensus.cc:2243] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.811421 22013 raft_consensus.cc:2272] T 985fb17e8948485b895e8edcd21bde2c P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.812049 22013 tablet_replica.cc:333] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e: stopping tablet replica
I20251210 21:06:21.812175 22013 raft_consensus.cc:2243] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.812355 22013 raft_consensus.cc:2272] T 571081fa9d9e4637b74c00b9a2035026 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.812773 22013 tablet_replica.cc:333] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e: stopping tablet replica
I20251210 21:06:21.812870 22013 raft_consensus.cc:2243] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.813017 22013 raft_consensus.cc:2272] T 576d8355b30b4cd0bed786091ba63f1d P 39db45a3749e4ec8a3dc2a2af04d659e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.813380 22013 tablet_replica.cc:333] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e: stopping tablet replica
I20251210 21:06:21.813473 22013 raft_consensus.cc:2243] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:21.813607 22013 raft_consensus.cc:2272] T 669859794b47427888e0c01cb869bad5 P 39db45a3749e4ec8a3dc2a2af04d659e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:21.827529 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:06:21.831182 22013 master.cc:561] Master@127.21.127.126:35949 shutting down...
W20251210 21:06:22.030169 27385 catalog_manager.cc:4712] TS 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 669859794b47427888e0c01cb869bad5: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
W20251210 21:06:24.592449 27385 catalog_manager.cc:4712] TS 555272d9a8304861a478617374fdd937 (127.21.127.65:33525): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 576d8355b30b4cd0bed786091ba63f1d: Network error: Client connection negotiation failed: client connection to 127.21.127.65:33525: connect: Connection refused (error 111)
I20251210 21:06:24.716583 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:24.716895 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:24.717036 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 26c4f150a2844647be598b98116f6cf6: stopping tablet replica
I20251210 21:06:24.769487 22013 master.cc:583] Master@127.21.127.126:35949 shutdown complete.
[ OK ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering (12786 ms)
[ RUN ] AutoRebalancerTest.TestHandlingFailedTservers
I20251210 21:06:24.783972 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:37443
I20251210 21:06:24.784574 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:24.787242 27891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:24.787350 27894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:24.787350 27892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:24.787535 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:24.788427 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:24.788507 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:24.788561 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400784788543 us; error 0 us; skew 500 ppm
I20251210 21:06:24.788820 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:24.790221 22013 webserver.cc:492] Webserver started at http://127.21.127.126:35853/ using document root <none> and password file <none>
I20251210 21:06:24.790462 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:24.790542 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:24.790675 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:24.791345 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "e86e248d51034501a7e71e2d1f3cea8b"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.793895 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:24.795578 27899 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.795938 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:24.796036 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "e86e248d51034501a7e71e2d1f3cea8b"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.796144 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:24.806049 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:24.806562 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:24.818184 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:37443
I20251210 21:06:24.818214 27950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:37443 every 8 connection(s)
I20251210 21:06:24.819519 27951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:24.823777 27951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b: Bootstrap starting.
I20251210 21:06:24.824968 27951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:24.826491 27951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b: No bootstrap required, opened a new log
I20251210 21:06:24.827101 27951 raft_consensus.cc:359] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e248d51034501a7e71e2d1f3cea8b" member_type: VOTER }
I20251210 21:06:24.827257 27951 raft_consensus.cc:385] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:24.827327 27951 raft_consensus.cc:740] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86e248d51034501a7e71e2d1f3cea8b, State: Initialized, Role: FOLLOWER
I20251210 21:06:24.827641 27951 consensus_queue.cc:260] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e248d51034501a7e71e2d1f3cea8b" member_type: VOTER }
I20251210 21:06:24.827811 27951 raft_consensus.cc:399] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:06:24.827905 27951 raft_consensus.cc:493] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:06:24.827991 27951 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:24.830027 27951 raft_consensus.cc:515] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e248d51034501a7e71e2d1f3cea8b" member_type: VOTER }
I20251210 21:06:24.830224 27951 leader_election.cc:304] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e86e248d51034501a7e71e2d1f3cea8b; no voters:
I20251210 21:06:24.830633 27951 leader_election.cc:290] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:06:24.830754 27954 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:24.831240 27954 raft_consensus.cc:697] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 1 LEADER]: Becoming Leader. State: Replica: e86e248d51034501a7e71e2d1f3cea8b, State: Running, Role: LEADER
I20251210 21:06:24.831501 27951 sys_catalog.cc:565] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:06:24.831491 27954 consensus_queue.cc:237] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e248d51034501a7e71e2d1f3cea8b" member_type: VOTER }
I20251210 21:06:24.832243 27956 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e86e248d51034501a7e71e2d1f3cea8b. Latest consensus state: current_term: 1 leader_uuid: "e86e248d51034501a7e71e2d1f3cea8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e248d51034501a7e71e2d1f3cea8b" member_type: VOTER } }
I20251210 21:06:24.832307 27955 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e86e248d51034501a7e71e2d1f3cea8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e248d51034501a7e71e2d1f3cea8b" member_type: VOTER } }
I20251210 21:06:24.832473 27956 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [sys.catalog]: This master's current role is: LEADER
I20251210 21:06:24.832559 27955 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [sys.catalog]: This master's current role is: LEADER
I20251210 21:06:24.833581 27958 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:06:24.835099 27958 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:06:24.836695 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:06:24.838284 27958 catalog_manager.cc:1357] Generated new cluster ID: 2e767475fdcd44abaace6dffeb95e70b
I20251210 21:06:24.838379 27958 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:06:24.848255 27958 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:06:24.849231 27958 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:06:24.856838 27958 catalog_manager.cc:6027] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b: Generated new TSK 0
I20251210 21:06:24.857113 27958 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:06:24.869853 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:24.872694 27972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:24.873060 27973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:24.873582 27975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:24.874251 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:24.874522 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:24.874586 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:24.874641 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400784874623 us; error 0 us; skew 500 ppm
I20251210 21:06:24.874799 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:24.876080 22013 webserver.cc:492] Webserver started at http://127.21.127.65:34855/ using document root <none> and password file <none>
I20251210 21:06:24.876327 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:24.876413 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:24.876650 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:24.877401 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "09bf6175fa244f069e6dde0efb8eda2c"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.880079 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20251210 21:06:24.881851 27980 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.882283 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:06:24.882413 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "09bf6175fa244f069e6dde0efb8eda2c"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.882540 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:24.894472 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:24.895040 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:24.895620 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:24.896560 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:24.896647 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.896739 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:24.896793 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.911553 28042 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:39101 every 8 connection(s)
I20251210 21:06:24.911553 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:39101
I20251210 21:06:24.913384 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:24.916356 28047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:24.917548 28048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:24.917654 28050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:24.918391 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:24.918720 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:24.918828 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:24.918912 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400784918873 us; error 0 us; skew 500 ppm
I20251210 21:06:24.919103 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:24.920785 22013 webserver.cc:492] Webserver started at http://127.21.127.66:33515/ using document root <none> and password file <none>
I20251210 21:06:24.921145 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:24.921269 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:24.921403 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:24.921734 28043 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37443
I20251210 21:06:24.921875 28043 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:24.922127 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.922300 28043 heartbeater.cc:507] Master 127.21.127.126:37443 requested a full tablet report, sending...
I20251210 21:06:24.923229 27916 ts_manager.cc:194] Registered new tserver with Master: 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:24.924445 27916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56000
I20251210 21:06:24.925076 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20251210 21:06:24.926714 28055 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.927013 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:24.927119 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.927235 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:24.940901 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:24.941491 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:24.942032 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:24.942878 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:24.942947 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.943040 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:24.943094 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.955596 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:41293
I20251210 21:06:24.955632 28117 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:41293 every 8 connection(s)
I20251210 21:06:24.957305 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:24.960275 28121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:24.960935 28122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:24.964845 22013 server_base.cc:1047] running on GCE node
W20251210 21:06:24.964926 28124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:24.965195 28118 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37443
I20251210 21:06:24.965318 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251210 21:06:24.965351 28118 heartbeater.cc:461] Registering TS with master...
W20251210 21:06:24.965397 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:24.965447 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400784965429 us; error 0 us; skew 500 ppm
I20251210 21:06:24.965569 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:24.965745 28118 heartbeater.cc:507] Master 127.21.127.126:37443 requested a full tablet report, sending...
I20251210 21:06:24.966749 22013 webserver.cc:492] Webserver started at http://127.21.127.67:34877/ using document root <none> and password file <none>
I20251210 21:06:24.966753 27916 ts_manager.cc:194] Registered new tserver with Master: febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:24.966984 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:24.967064 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:24.967204 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:24.967790 27916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56008
I20251210 21:06:24.967901 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "05b9cf62b62b446d8241939224cd3b22"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.970451 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251210 21:06:24.972146 28129 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.972535 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:06:24.972648 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "05b9cf62b62b446d8241939224cd3b22"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:24.972760 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:24.987637 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:24.988204 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:24.988750 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:24.989620 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:24.989691 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:24.989784 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:24.989835 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:25.001726 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:36489
I20251210 21:06:25.001749 28191 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:36489 every 8 connection(s)
I20251210 21:06:25.008786 28192 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37443
I20251210 21:06:25.008941 28192 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:25.009375 28192 heartbeater.cc:507] Master 127.21.127.126:37443 requested a full tablet report, sending...
I20251210 21:06:25.010211 27916 ts_manager.cc:194] Registered new tserver with Master: 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:25.010511 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007933493s
I20251210 21:06:25.011078 27916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56024
I20251210 21:06:25.925874 28043 heartbeater.cc:499] Master 127.21.127.126:37443 was elected leader, sending a full tablet report...
I20251210 21:06:25.969102 28118 heartbeater.cc:499] Master 127.21.127.126:37443 was elected leader, sending a full tablet report...
I20251210 21:06:26.012540 28192 heartbeater.cc:499] Master 127.21.127.126:37443 was elected leader, sending a full tablet report...
I20251210 21:06:26.038648 22013 test_util.cc:276] Using random seed: 1723658095
I20251210 21:06:26.051862 27916 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43954:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:06:26.052655 27916 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:06:26.075129 28157 tablet_service.cc:1505] Processing CreateTablet for tablet 8244713e417e41c999b1a3d56357bb81 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:26.075629 28157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8244713e417e41c999b1a3d56357bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.077579 28083 tablet_service.cc:1505] Processing CreateTablet for tablet 8244713e417e41c999b1a3d56357bb81 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:26.077703 28081 tablet_service.cc:1505] Processing CreateTablet for tablet 2fc72d6b1a4046f38bea511c85bfa3e3 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:26.077983 28083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8244713e417e41c999b1a3d56357bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.077677 28082 tablet_service.cc:1505] Processing CreateTablet for tablet 23f151792969458783a4b5eece7a02d5 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:26.078333 28082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f151792969458783a4b5eece7a02d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.079207 28155 tablet_service.cc:1505] Processing CreateTablet for tablet 2fc72d6b1a4046f38bea511c85bfa3e3 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:26.079665 28155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc72d6b1a4046f38bea511c85bfa3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.079692 28008 tablet_service.cc:1505] Processing CreateTablet for tablet 8244713e417e41c999b1a3d56357bb81 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:26.080052 28008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8244713e417e41c999b1a3d56357bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.082696 28156 tablet_service.cc:1505] Processing CreateTablet for tablet 23f151792969458783a4b5eece7a02d5 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:26.083128 28156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f151792969458783a4b5eece7a02d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.083736 28081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc72d6b1a4046f38bea511c85bfa3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.086390 28080 tablet_service.cc:1505] Processing CreateTablet for tablet 257ae5d4e71d4a00859a8047e798516d (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:26.086795 28080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257ae5d4e71d4a00859a8047e798516d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.084506 28007 tablet_service.cc:1505] Processing CreateTablet for tablet 23f151792969458783a4b5eece7a02d5 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:26.087714 28007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f151792969458783a4b5eece7a02d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.087785 28154 tablet_service.cc:1505] Processing CreateTablet for tablet 257ae5d4e71d4a00859a8047e798516d (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:26.088189 28154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257ae5d4e71d4a00859a8047e798516d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.084813 28006 tablet_service.cc:1505] Processing CreateTablet for tablet 2fc72d6b1a4046f38bea511c85bfa3e3 (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:26.091877 28006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc72d6b1a4046f38bea511c85bfa3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.085131 28005 tablet_service.cc:1505] Processing CreateTablet for tablet 257ae5d4e71d4a00859a8047e798516d (DEFAULT_TABLE table=test-workload [id=28f46c31cfb2414c947c573cadae23a4]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:26.095906 28005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257ae5d4e71d4a00859a8047e798516d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:26.101404 28214 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:26.103204 28214 tablet_bootstrap.cc:654] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.104399 28213 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:26.106338 28213 tablet_bootstrap.cc:654] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.106632 28212 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:26.107556 28214 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: No bootstrap required, opened a new log
I20251210 21:06:26.107707 28214 ts_tablet_manager.cc:1403] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251210 21:06:26.108223 28213 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: No bootstrap required, opened a new log
I20251210 21:06:26.108346 28213 ts_tablet_manager.cc:1403] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:26.108356 28212 tablet_bootstrap.cc:654] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.108467 28214 raft_consensus.cc:359] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.108655 28214 raft_consensus.cc:385] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.108759 28214 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.109015 28213 raft_consensus.cc:359] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.109236 28213 raft_consensus.cc:385] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.109004 28214 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.109318 28213 raft_consensus.cc:740] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.109524 28213 consensus_queue.cc:260] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.109927 28214 ts_tablet_manager.cc:1434] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:26.110116 28212 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: No bootstrap required, opened a new log
I20251210 21:06:26.110273 28212 ts_tablet_manager.cc:1403] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:26.110309 28214 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:26.110926 28212 raft_consensus.cc:359] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.111120 28212 raft_consensus.cc:385] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.111219 28212 raft_consensus.cc:740] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.111529 28212 consensus_queue.cc:260] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.112134 28214 tablet_bootstrap.cc:654] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.115289 28212 ts_tablet_manager.cc:1434] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251210 21:06:26.115835 28213 ts_tablet_manager.cc:1434] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20251210 21:06:26.116146 28213 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:26.116216 28212 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:26.117710 28213 tablet_bootstrap.cc:654] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.117710 28212 tablet_bootstrap.cc:654] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.119767 28214 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: No bootstrap required, opened a new log
I20251210 21:06:26.119900 28214 ts_tablet_manager.cc:1403] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20251210 21:06:26.120455 28214 raft_consensus.cc:359] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.120596 28214 raft_consensus.cc:385] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.120669 28214 raft_consensus.cc:740] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.120862 28214 consensus_queue.cc:260] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.121296 28214 ts_tablet_manager.cc:1434] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:26.121589 28214 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:26.124004 28214 tablet_bootstrap.cc:654] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.126470 28214 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: No bootstrap required, opened a new log
I20251210 21:06:26.126600 28214 ts_tablet_manager.cc:1403] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:06:26.127156 28214 raft_consensus.cc:359] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.127318 28214 raft_consensus.cc:385] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.127378 28214 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.127607 28214 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.128077 28214 ts_tablet_manager.cc:1434] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:26.128415 28214 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:26.128564 28212 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: No bootstrap required, opened a new log
I20251210 21:06:26.128566 28213 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: No bootstrap required, opened a new log
I20251210 21:06:26.128767 28212 ts_tablet_manager.cc:1403] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20251210 21:06:26.128805 28213 ts_tablet_manager.cc:1403] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20251210 21:06:26.129484 28213 raft_consensus.cc:359] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.129484 28212 raft_consensus.cc:359] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.129705 28213 raft_consensus.cc:385] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.129781 28212 raft_consensus.cc:385] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.129818 28213 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.129874 28212 raft_consensus.cc:740] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.130157 28213 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.130163 28212 consensus_queue.cc:260] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.131222 28212 ts_tablet_manager.cc:1434] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:26.131222 28213 ts_tablet_manager.cc:1434] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:06:26.131563 28212 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:26.131563 28213 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:26.133175 28213 tablet_bootstrap.cc:654] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.133175 28212 tablet_bootstrap.cc:654] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.135277 28214 tablet_bootstrap.cc:654] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.137128 28214 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: No bootstrap required, opened a new log
I20251210 21:06:26.137276 28214 ts_tablet_manager.cc:1403] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20251210 21:06:26.137286 28212 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: No bootstrap required, opened a new log
I20251210 21:06:26.137455 28212 ts_tablet_manager.cc:1403] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251210 21:06:26.137306 28213 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: No bootstrap required, opened a new log
I20251210 21:06:26.138031 28214 raft_consensus.cc:359] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.138031 28212 raft_consensus.cc:359] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.138245 28213 ts_tablet_manager.cc:1403] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251210 21:06:26.138263 28214 raft_consensus.cc:385] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.138310 28212 raft_consensus.cc:385] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.138347 28214 raft_consensus.cc:740] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.138360 28212 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.138561 28212 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.138561 28214 consensus_queue.cc:260] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.138975 28213 raft_consensus.cc:359] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.139063 28214 ts_tablet_manager.cc:1434] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:26.139066 28212 ts_tablet_manager.cc:1434] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:26.139114 28213 raft_consensus.cc:385] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.139201 28213 raft_consensus.cc:740] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.139374 28213 consensus_queue.cc:260] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.139804 28212 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:26.139827 28213 ts_tablet_manager.cc:1434] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:26.140194 28213 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:26.141757 28213 tablet_bootstrap.cc:654] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.142709 28212 tablet_bootstrap.cc:654] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:26.145485 28213 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: No bootstrap required, opened a new log
I20251210 21:06:26.145617 28213 ts_tablet_manager.cc:1403] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251210 21:06:26.146222 28213 raft_consensus.cc:359] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.146356 28213 raft_consensus.cc:385] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.146410 28213 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.146636 28213 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.147030 28213 ts_tablet_manager.cc:1434] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:26.148319 28212 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: No bootstrap required, opened a new log
I20251210 21:06:26.148428 28212 ts_tablet_manager.cc:1403] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20251210 21:06:26.149045 28212 raft_consensus.cc:359] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.149168 28212 raft_consensus.cc:385] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:26.149228 28212 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:26.149425 28212 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.149858 28212 ts_tablet_manager.cc:1434] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20251210 21:06:26.163784 28044 tablet.cc:2378] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251210 21:06:26.173547 28219 raft_consensus.cc:493] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:26.173736 28219 raft_consensus.cc:515] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.174649 28219 leader_election.cc:290] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:26.180982 28093 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:26.180996 28018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c" is_pre_election: true
I20251210 21:06:26.181298 28093 raft_consensus.cc:2468] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 0.
I20251210 21:06:26.181371 28018 raft_consensus.cc:2468] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 0.
I20251210 21:06:26.181778 28131 leader_election.cc:304] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:26.182103 28219 raft_consensus.cc:2804] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:26.182219 28219 raft_consensus.cc:493] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:26.182310 28219 raft_consensus.cc:3060] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.184478 28219 raft_consensus.cc:515] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.185078 28219 leader_election.cc:290] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 election: Requested vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:26.185505 28018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c"
I20251210 21:06:26.185514 28093 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:26.185714 28018 raft_consensus.cc:3060] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.185724 28093 raft_consensus.cc:3060] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.187952 28018 raft_consensus.cc:2468] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 1.
I20251210 21:06:26.188171 28093 raft_consensus.cc:2468] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 1.
I20251210 21:06:26.188380 28133 leader_election.cc:304] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, 09bf6175fa244f069e6dde0efb8eda2c; no voters:
I20251210 21:06:26.188716 28219 raft_consensus.cc:2804] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:26.189168 28219 raft_consensus.cc:697] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 LEADER]: Becoming Leader. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Running, Role: LEADER
I20251210 21:06:26.189440 28219 consensus_queue.cc:237] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.192037 27916 catalog_manager.cc:5654] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67). New cstate: current_term: 1 leader_uuid: "05b9cf62b62b446d8241939224cd3b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:26.236140 28218 raft_consensus.cc:493] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:26.236371 28218 raft_consensus.cc:515] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.237298 28218 leader_election.cc:290] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:26.243647 28093 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:26.243647 28167 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22" is_pre_election: true
I20251210 21:06:26.244021 28093 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 0.
I20251210 21:06:26.244050 28167 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 0.
I20251210 21:06:26.244516 27984 leader_election.cc:304] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:26.244840 28218 raft_consensus.cc:2804] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:26.244994 28218 raft_consensus.cc:493] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:26.245086 28218 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.245366 28219 raft_consensus.cc:493] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:26.245528 28219 raft_consensus.cc:515] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.246260 28219 leader_election.cc:290] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:26.246737 28093 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:26.246949 28093 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 0.
I20251210 21:06:26.247298 28018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c" is_pre_election: true
I20251210 21:06:26.247360 28131 leader_election.cc:304] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:26.247515 28018 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 0.
I20251210 21:06:26.247695 28219 raft_consensus.cc:2804] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:26.247849 28219 raft_consensus.cc:493] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:26.247960 28219 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.248121 28218 raft_consensus.cc:515] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.248682 28218 leader_election.cc:290] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 1 election: Requested vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:26.249167 28093 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:26.249251 28167 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22"
I20251210 21:06:26.249356 28093 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.249444 28167 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.250806 28219 raft_consensus.cc:515] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.251282 28093 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 1.
I20251210 21:06:26.251821 27982 leader_election.cc:304] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:26.251909 28018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c"
I20251210 21:06:26.252005 28092 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:26.252104 28018 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.252218 28092 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.252255 28218 raft_consensus.cc:2804] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:26.252332 28167 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 1.
I20251210 21:06:26.254633 28092 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 1.
I20251210 21:06:26.254917 28219 leader_election.cc:290] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 election: Requested vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:26.255090 28131 leader_election.cc:304] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:26.255250 28018 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 1.
I20251210 21:06:26.255302 28218 raft_consensus.cc:697] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 LEADER]: Becoming Leader. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Running, Role: LEADER
I20251210 21:06:26.255364 28219 raft_consensus.cc:2804] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:26.255563 28219 raft_consensus.cc:697] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 LEADER]: Becoming Leader. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Running, Role: LEADER
I20251210 21:06:26.255676 28218 consensus_queue.cc:237] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.255822 28219 consensus_queue.cc:237] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:26.258432 27916 catalog_manager.cc:5654] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65). New cstate: current_term: 1 leader_uuid: "09bf6175fa244f069e6dde0efb8eda2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:26.258466 27914 catalog_manager.cc:5654] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67). New cstate: current_term: 1 leader_uuid: "05b9cf62b62b446d8241939224cd3b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:26.259835 28220 raft_consensus.cc:493] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:26.260035 28220 raft_consensus.cc:515] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.260833 28220 leader_election.cc:290] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489), 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:26.268187 28167 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22" is_pre_election: true
I20251210 21:06:26.268476 28167 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate febf21f9ae84480f8b4c3da2ae6c4706 in term 0.
I20251210 21:06:26.268886 28059 leader_election.cc:304] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:26.269040 28018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c" is_pre_election: true
I20251210 21:06:26.269250 28220 raft_consensus.cc:2804] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:26.269325 28018 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate febf21f9ae84480f8b4c3da2ae6c4706 in term 0.
I20251210 21:06:26.269407 28220 raft_consensus.cc:493] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:26.269510 28220 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.272716 28220 raft_consensus.cc:515] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.273270 28220 leader_election.cc:290] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 1 election: Requested vote from peers 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489), 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:26.273685 28167 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22"
I20251210 21:06:26.273765 28018 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c"
I20251210 21:06:26.273860 28167 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.273977 28018 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:26.275916 28018 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate febf21f9ae84480f8b4c3da2ae6c4706 in term 1.
I20251210 21:06:26.275919 28167 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate febf21f9ae84480f8b4c3da2ae6c4706 in term 1.
I20251210 21:06:26.276338 28059 leader_election.cc:304] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:26.276641 28220 raft_consensus.cc:2804] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:26.277066 28220 raft_consensus.cc:697] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 LEADER]: Becoming Leader. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Running, Role: LEADER
I20251210 21:06:26.277388 28220 consensus_queue.cc:237] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:26.280040 27914 catalog_manager.cc:5654] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 reported cstate change: term changed from 0 to 1, leader changed from <none> to febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66). New cstate: current_term: 1 leader_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:26.318939 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:06:26.326367 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:26.326787 22013 tablet_replica.cc:333] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: stopping tablet replica
I20251210 21:06:26.327004 22013 raft_consensus.cc:2243] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.327178 22013 raft_consensus.cc:2272] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.327958 22013 tablet_replica.cc:333] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: stopping tablet replica
I20251210 21:06:26.328105 22013 raft_consensus.cc:2243] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.328275 22013 raft_consensus.cc:2272] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.328867 22013 tablet_replica.cc:333] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: stopping tablet replica
I20251210 21:06:26.329010 22013 raft_consensus.cc:2243] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:26.329280 22013 pending_rounds.cc:70] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Trying to abort 1 pending ops.
I20251210 21:06:26.329348 22013 pending_rounds.cc:77] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081620505632768 op_type: NO_OP noop_request { }
I20251210 21:06:26.329473 22013 raft_consensus.cc:2889] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:26.329577 22013 raft_consensus.cc:2272] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.330304 22013 tablet_replica.cc:333] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: stopping tablet replica
I20251210 21:06:26.330440 22013 raft_consensus.cc:2243] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.330571 22013 raft_consensus.cc:2272] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.345692 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:06:26.349771 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
I20251210 21:06:26.357393 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:26.357833 22013 tablet_replica.cc:333] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: stopping tablet replica
I20251210 21:06:26.358065 22013 raft_consensus.cc:2243] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:26.358307 22013 pending_rounds.cc:70] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Trying to abort 1 pending ops.
I20251210 21:06:26.358361 22013 pending_rounds.cc:77] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081620594655232 op_type: NO_OP noop_request { }
I20251210 21:06:26.358474 22013 raft_consensus.cc:2889] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:26.358556 22013 raft_consensus.cc:2272] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.359359 22013 tablet_replica.cc:333] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: stopping tablet replica
I20251210 21:06:26.359548 22013 raft_consensus.cc:2243] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.359688 22013 raft_consensus.cc:2272] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.360266 22013 tablet_replica.cc:333] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: stopping tablet replica
I20251210 21:06:26.360384 22013 raft_consensus.cc:2243] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.360494 22013 raft_consensus.cc:2272] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.361039 22013 tablet_replica.cc:333] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: stopping tablet replica
I20251210 21:06:26.361147 22013 raft_consensus.cc:2243] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.361259 22013 raft_consensus.cc:2272] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.376549 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:06:26.380153 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:06:26.388631 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:26.389016 22013 tablet_replica.cc:333] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: stopping tablet replica
I20251210 21:06:26.389214 22013 raft_consensus.cc:2243] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.389397 22013 raft_consensus.cc:2272] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.390126 22013 tablet_replica.cc:333] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: stopping tablet replica
I20251210 21:06:26.390280 22013 raft_consensus.cc:2243] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:26.390498 22013 pending_rounds.cc:70] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Trying to abort 1 pending ops.
I20251210 21:06:26.390558 22013 pending_rounds.cc:77] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081620506521600 op_type: NO_OP noop_request { }
I20251210 21:06:26.390667 22013 raft_consensus.cc:2889] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:26.390751 22013 raft_consensus.cc:2272] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.392050 22013 tablet_replica.cc:333] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: stopping tablet replica
I20251210 21:06:26.392172 22013 raft_consensus.cc:2243] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:26.392383 22013 pending_rounds.cc:70] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Trying to abort 1 pending ops.
I20251210 21:06:26.392448 22013 pending_rounds.cc:77] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081620234264576 op_type: NO_OP noop_request { }
I20251210 21:06:26.392558 22013 raft_consensus.cc:2889] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:26.392661 22013 raft_consensus.cc:2272] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.393435 22013 tablet_replica.cc:333] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: stopping tablet replica
I20251210 21:06:26.393558 22013 raft_consensus.cc:2243] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:26.393673 22013 raft_consensus.cc:2272] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:26.398442 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:06:26.403208 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:26.405997 28244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:26.406005 28245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:26.406492 28247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:26.406509 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:26.406821 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:26.406886 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:26.406932 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400786406929 us; error 0 us; skew 500 ppm
I20251210 21:06:26.407083 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:26.408313 22013 webserver.cc:492] Webserver started at http://127.21.127.68:44669/ using document root <none> and password file <none>
I20251210 21:06:26.408522 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:26.408591 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:26.408696 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:26.409350 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "fb19a7f9aad3441196a840e811c68e82"
format_stamp: "Formatted at 2025-12-10 21:06:26 on dist-test-slave-9pq8"
I20251210 21:06:26.411985 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:26.413645 28252 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:26.414070 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:26.414180 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "fb19a7f9aad3441196a840e811c68e82"
format_stamp: "Formatted at 2025-12-10 21:06:26 on dist-test-slave-9pq8"
I20251210 21:06:26.414312 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:26.424911 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:26.425478 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:26.426072 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:26.426961 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:26.427047 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:26.427143 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:26.427191 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:26.440310 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:43571
I20251210 21:06:26.440336 28314 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:43571 every 8 connection(s)
I20251210 21:06:26.449874 28315 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37443
I20251210 21:06:26.450054 28315 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:26.450446 28315 heartbeater.cc:507] Master 127.21.127.126:37443 requested a full tablet report, sending...
I20251210 21:06:26.451472 27914 ts_manager.cc:194] Registered new tserver with Master: fb19a7f9aad3441196a840e811c68e82 (127.21.127.68:43571)
I20251210 21:06:26.452311 27914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43964
W20251210 21:06:26.839046 27964 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:26.840338 27903 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20251210 21:06:26.841506 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:26.841614 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:26.843204 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:26.843295 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:26.844765 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:26.844863 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
I20251210 21:06:27.453598 28315 heartbeater.cc:499] Master 127.21.127.126:37443 was elected leader, sending a full tablet report...
W20251210 21:06:27.849035 27964 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.21.127.66:41293: connect: Connection refused (error 111)
W20251210 21:06:27.852634 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.66:41293: connect: Connection refused (error 111)
W20251210 21:06:27.852743 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.66:41293: connect: Connection refused (error 111)
W20251210 21:06:27.854252 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:27.854329 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:27.855717 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:27.855787 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:28.859697 27964 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:28.868736 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:28.868844 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:28.870644 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:28.870726 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:28.872437 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:28.872516 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:29.876426 27964 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:29.880461 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:29.880558 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111)
W20251210 21:06:29.882121 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:29.882205 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.67:36489: connect: Connection refused (error 111)
W20251210 21:06:29.883726 27964 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.21.127.66:41293: connect: Connection refused (error 111)
W20251210 21:06:29.883805 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.21.127.66:41293: connect: Connection refused (error 111)
W20251210 21:06:30.886096 27964 auto_rebalancer.cc:249] could not retrieve auto-rebalancing replica moves: Not found: table 28f46c31cfb2414c947c573cadae23a4: could not find any suitable replica to move from server 05b9cf62b62b446d8241939224cd3b22 to server fb19a7f9aad3441196a840e811c68e82
W20251210 21:06:31.887027 27964 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 05b9cf62b62b446d8241939224cd3b22 not available for placement
I20251210 21:06:31.963655 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:31.966702 28330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:31.966836 28331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:31.971864 28333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:31.977833 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:31.978678 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:31.978760 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:31.978802 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400791978782 us; error 0 us; skew 500 ppm
I20251210 21:06:31.978958 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:31.980408 22013 webserver.cc:492] Webserver started at http://127.21.127.65:34855/ using document root <none> and password file <none>
I20251210 21:06:31.980656 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:31.980720 22013 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251210 21:06:31.982678 22013 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:31.984336 28338 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:31.984784 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:31.984905 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "09bf6175fa244f069e6dde0efb8eda2c"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:31.985086 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:32.004689 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:32.005290 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:32.005863 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:32.007385 28345 ts_tablet_manager.cc:548] Loading tablet metadata (0/4 complete)
I20251210 21:06:32.016050 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251210 21:06:32.016172 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s user 0.001s sys 0.000s
I20251210 21:06:32.016311 22013 ts_tablet_manager.cc:600] Registering tablets (0/4 complete)
I20251210 21:06:32.018378 28345 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:32.022583 28345 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251210 21:06:32.023093 28345 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap complete.
I20251210 21:06:32.023267 22013 ts_tablet_manager.cc:616] Registered 4 tablets
I20251210 21:06:32.023336 28345 ts_tablet_manager.cc:1403] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:06:32.023356 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.007s user 0.003s sys 0.000s
I20251210 21:06:32.024030 28345 raft_consensus.cc:359] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.024295 28345 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.024560 28345 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.025277 28345 ts_tablet_manager.cc:1434] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251210 21:06:32.025576 28345 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:32.029266 28345 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.029690 28345 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap complete.
I20251210 21:06:32.029927 28345 ts_tablet_manager.cc:1403] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:32.030546 28345 raft_consensus.cc:359] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.030704 28345 raft_consensus.cc:740] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.030916 28345 consensus_queue.cc:260] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.031325 28345 ts_tablet_manager.cc:1434] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:32.031589 28345 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:32.035607 28345 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.035992 28345 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap complete.
I20251210 21:06:32.036250 28345 ts_tablet_manager.cc:1403] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20251210 21:06:32.036842 28345 raft_consensus.cc:359] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:32.036998 28345 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.037231 28345 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:32.037601 28345 ts_tablet_manager.cc:1434] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:32.037811 28345 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap starting.
I20251210 21:06:32.041464 28345 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.041952 28345 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Bootstrap complete.
I20251210 21:06:32.042277 28345 ts_tablet_manager.cc:1403] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251210 21:06:32.043146 28345 raft_consensus.cc:359] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.043375 28345 raft_consensus.cc:740] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.043839 28406 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:39101 every 8 connection(s)
I20251210 21:06:32.043840 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:39101
I20251210 21:06:32.043771 28345 consensus_queue.cc:260] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.044361 28345 ts_tablet_manager.cc:1434] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:32.045436 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:32.048596 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:32.049188 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:32.053007 22013 server_base.cc:1047] running on GCE node
W20251210 21:06:32.053186 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:32.053531 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:32.053613 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:32.053654 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400792053639 us; error 0 us; skew 500 ppm
I20251210 21:06:32.053771 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:32.054997 22013 webserver.cc:492] Webserver started at http://127.21.127.66:33515/ using document root <none> and password file <none>
I20251210 21:06:32.055229 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:32.055289 22013 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251210 21:06:32.057529 22013 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:32.057739 28407 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37443
I20251210 21:06:32.057893 28407 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:32.058372 28407 heartbeater.cc:507] Master 127.21.127.126:37443 requested a full tablet report, sending...
I20251210 21:06:32.059474 28419 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:32.059846 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251210 21:06:32.059948 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:32.060050 27914 ts_manager.cc:194] Re-registered known tserver with Master: 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:32.060078 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:32.062078 27914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43970
I20251210 21:06:32.080332 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:32.080884 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:32.081455 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:32.090803 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251210 21:06:32.090914 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s user 0.001s sys 0.000s
I20251210 21:06:32.092998 28426 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:32.096846 28426 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.097245 28426 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap complete.
I20251210 21:06:32.097493 28426 ts_tablet_manager.cc:1403] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:06:32.098146 22013 ts_tablet_manager.cc:616] Registered 4 tablets
I20251210 21:06:32.098127 28426 raft_consensus.cc:359] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.098290 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.007s user 0.007s sys 0.000s
I20251210 21:06:32.098321 28426 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.098575 28426 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.099395 28426 ts_tablet_manager.cc:1434] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:32.099690 28426 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:32.104390 28426 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.104753 28426 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap complete.
I20251210 21:06:32.104990 28426 ts_tablet_manager.cc:1403] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251210 21:06:32.105640 28426 raft_consensus.cc:359] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.105811 28426 raft_consensus.cc:740] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.106055 28426 consensus_queue.cc:260] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.106477 28426 ts_tablet_manager.cc:1434] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:32.106698 28426 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:32.111069 28426 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251210 21:06:32.111601 28426 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap complete.
I20251210 21:06:32.111924 28426 ts_tablet_manager.cc:1403] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.002s
I20251210 21:06:32.112676 28426 raft_consensus.cc:359] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:32.112944 28426 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.113272 28426 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:32.113732 28426 ts_tablet_manager.cc:1434] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:06:32.114042 28426 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap starting.
I20251210 21:06:32.116226 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:41293
I20251210 21:06:32.116235 28486 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:41293 every 8 connection(s)
I20251210 21:06:32.118078 28426 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.118088 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:06:32.118525 28426 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Bootstrap complete.
I20251210 21:06:32.118825 28426 ts_tablet_manager.cc:1403] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251210 21:06:32.119792 28426 raft_consensus.cc:359] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.120097 28426 raft_consensus.cc:740] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.120404 28426 consensus_queue.cc:260] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.120968 28426 ts_tablet_manager.cc:1434] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20251210 21:06:32.121295 28491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:32.122660 28492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:32.123757 22013 server_base.cc:1047] running on GCE node
W20251210 21:06:32.124437 28494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:32.124887 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:32.124966 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:32.125032 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400792125004 us; error 0 us; skew 500 ppm
I20251210 21:06:32.125198 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:32.126662 22013 webserver.cc:492] Webserver started at http://127.21.127.67:34877/ using document root <none> and password file <none>
I20251210 21:06:32.126940 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:32.127012 22013 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251210 21:06:32.129551 22013 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:32.129779 28487 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37443
I20251210 21:06:32.129917 28487 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:32.130324 28487 heartbeater.cc:507] Master 127.21.127.126:37443 requested a full tablet report, sending...
I20251210 21:06:32.131275 28499 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:32.131668 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:32.131772 27914 ts_manager.cc:194] Re-registered known tserver with Master: febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:32.131786 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "05b9cf62b62b446d8241939224cd3b22"
format_stamp: "Formatted at 2025-12-10 21:06:24 on dist-test-slave-9pq8"
I20251210 21:06:32.131994 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:32.133924 27914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43984
I20251210 21:06:32.135172 28487 heartbeater.cc:499] Master 127.21.127.126:37443 was elected leader, sending a full tablet report...
I20251210 21:06:32.142311 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:32.142817 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:32.143400 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:32.153549 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251210 21:06:32.153677 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.001s
I20251210 21:06:32.155891 28506 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:32.160086 28506 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.160455 28506 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Bootstrap complete.
I20251210 21:06:32.160718 28506 ts_tablet_manager.cc:1403] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:06:32.160898 22013 ts_tablet_manager.cc:616] Registered 4 tablets
I20251210 21:06:32.160990 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.007s user 0.005s sys 0.001s
I20251210 21:06:32.161654 28506 raft_consensus.cc:359] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.161818 28506 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.162081 28506 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.162837 28506 ts_tablet_manager.cc:1434] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:32.163123 28506 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:32.168087 28506 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251210 21:06:32.168639 28506 tablet_bootstrap.cc:492] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap complete.
I20251210 21:06:32.168955 28506 ts_tablet_manager.cc:1403] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251210 21:06:32.169914 28506 raft_consensus.cc:359] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.170226 28506 raft_consensus.cc:740] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.170565 28506 consensus_queue.cc:260] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.171132 28506 ts_tablet_manager.cc:1434] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:32.171458 28506 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:32.176592 28506 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:32.176947 28506 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap complete.
I20251210 21:06:32.177210 28506 ts_tablet_manager.cc:1403] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20251210 21:06:32.178000 28506 raft_consensus.cc:359] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:32.178241 28506 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.178488 28506 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:32.178923 28506 ts_tablet_manager.cc:1434] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:06:32.179206 28506 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap starting.
I20251210 21:06:32.183786 28506 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251210 21:06:32.184239 28506 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Bootstrap complete.
I20251210 21:06:32.184544 28506 ts_tablet_manager.cc:1403] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251210 21:06:32.185297 28506 raft_consensus.cc:359] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.185521 28506 raft_consensus.cc:740] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Initialized, Role: FOLLOWER
I20251210 21:06:32.185727 28506 consensus_queue.cc:260] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:32.186158 28506 ts_tablet_manager.cc:1434] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:06:32.199750 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:36489
I20251210 21:06:32.199771 28567 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:36489 every 8 connection(s)
I20251210 21:06:32.206410 28568 heartbeater.cc:344] Connected to a master server at 127.21.127.126:37443
I20251210 21:06:32.206574 28568 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:32.206943 28568 heartbeater.cc:507] Master 127.21.127.126:37443 requested a full tablet report, sending...
I20251210 21:06:32.208312 27914 ts_manager.cc:194] Re-registered known tserver with Master: 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:32.210109 27914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43990
W20251210 21:06:32.896657 27964 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Replica 09bf6175fa244f069e6dde0efb8eda2c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } }
W20251210 21:06:32.898407 27964 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 257ae5d4e71d4a00859a8047e798516d, TS febf21f9ae84480f8b4c3da2ae6c4706 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
W20251210 21:06:32.898515 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 257ae5d4e71d4a00859a8047e798516d, TS febf21f9ae84480f8b4c3da2ae6c4706 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
W20251210 21:06:32.904814 27964 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 8244713e417e41c999b1a3d56357bb81, TS 05b9cf62b62b446d8241939224cd3b22 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
W20251210 21:06:32.904935 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 8244713e417e41c999b1a3d56357bb81, TS 05b9cf62b62b446d8241939224cd3b22 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
W20251210 21:06:32.910923 27964 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 23f151792969458783a4b5eece7a02d5, TS 09bf6175fa244f069e6dde0efb8eda2c -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
W20251210 21:06:32.911113 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 23f151792969458783a4b5eece7a02d5, TS 09bf6175fa244f069e6dde0efb8eda2c -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.063648 28407 heartbeater.cc:499] Master 127.21.127.126:37443 was elected leader, sending a full tablet report...
I20251210 21:06:33.211498 28568 heartbeater.cc:499] Master 127.21.127.126:37443 was elected leader, sending a full tablet report...
I20251210 21:06:33.224428 28575 raft_consensus.cc:493] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:33.224645 28575 raft_consensus.cc:515] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:33.225494 28575 leader_election.cc:290] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:33.231788 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22" is_pre_election: true
I20251210 21:06:33.231789 28461 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:33.232034 28541 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 1.
I20251210 21:06:33.232118 28461 raft_consensus.cc:2410] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09bf6175fa244f069e6dde0efb8eda2c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.232478 28342 leader_election.cc:304] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, 09bf6175fa244f069e6dde0efb8eda2c; no voters:
I20251210 21:06:33.232702 28575 raft_consensus.cc:2804] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:06:33.232774 28575 raft_consensus.cc:493] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:33.232841 28575 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.234812 28575 raft_consensus.cc:515] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:33.235347 28575 leader_election.cc:290] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 election: Requested vote from peers 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:33.235786 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22"
I20251210 21:06:33.235787 28461 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:33.235982 28461 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.235982 28541 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.238935 28541 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 2.
I20251210 21:06:33.239028 28461 raft_consensus.cc:2410] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 09bf6175fa244f069e6dde0efb8eda2c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.239352 28342 leader_election.cc:304] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, 09bf6175fa244f069e6dde0efb8eda2c; no voters:
I20251210 21:06:33.239662 28575 raft_consensus.cc:2804] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:06:33.240146 28575 raft_consensus.cc:697] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Becoming Leader. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Running, Role: LEADER
I20251210 21:06:33.240399 28575 consensus_queue.cc:237] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:33.243062 27914 catalog_manager.cc:5654] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c reported cstate change: term changed from 1 to 2, leader changed from febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66) to 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65). New cstate: current_term: 2 leader_uuid: "09bf6175fa244f069e6dde0efb8eda2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:33.309320 28584 raft_consensus.cc:493] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:33.309510 28584 raft_consensus.cc:515] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:33.310319 28584 leader_election.cc:290] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489), 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:33.316253 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22" is_pre_election: true
I20251210 21:06:33.316365 28380 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c" is_pre_election: true
I20251210 21:06:33.316483 28541 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate febf21f9ae84480f8b4c3da2ae6c4706 in term 2.
I20251210 21:06:33.316826 28423 leader_election.cc:304] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:33.317101 28584 raft_consensus.cc:2804] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251210 21:06:33.317193 28584 raft_consensus.cc:493] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:33.317266 28584 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.319133 28584 raft_consensus.cc:515] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:33.319720 28584 leader_election.cc:290] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 3 election: Requested vote from peers 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489), 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:33.320183 28380 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c"
I20251210 21:06:33.320190 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8244713e417e41c999b1a3d56357bb81" candidate_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22"
I20251210 21:06:33.320394 28541 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.322266 28541 raft_consensus.cc:2468] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate febf21f9ae84480f8b4c3da2ae6c4706 in term 3.
I20251210 21:06:33.322602 28423 leader_election.cc:304] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters: 09bf6175fa244f069e6dde0efb8eda2c
I20251210 21:06:33.322863 28584 raft_consensus.cc:2804] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:06:33.323239 28584 raft_consensus.cc:697] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 LEADER]: Becoming Leader. State: Replica: febf21f9ae84480f8b4c3da2ae6c4706, State: Running, Role: LEADER
I20251210 21:06:33.323523 28584 consensus_queue.cc:237] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:33.325991 27914 catalog_manager.cc:5654] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 reported cstate change: term changed from 2 to 3, leader changed from 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65) to febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66). New cstate: current_term: 3 leader_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:33.353657 28589 raft_consensus.cc:493] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:33.353839 28589 raft_consensus.cc:515] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.354707 28589 leader_election.cc:290] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:33.359283 28575 raft_consensus.cc:493] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:33.359496 28575 raft_consensus.cc:515] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.360172 28575 leader_election.cc:290] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:33.360582 28461 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:33.360607 28380 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c" is_pre_election: true
I20251210 21:06:33.360715 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:33.360607 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22" is_pre_election: true
I20251210 21:06:33.360924 28380 raft_consensus.cc:2410] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05b9cf62b62b446d8241939224cd3b22 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.360877 28460 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 1.
I20251210 21:06:33.361006 28541 raft_consensus.cc:2410] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09bf6175fa244f069e6dde0efb8eda2c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.360857 28461 raft_consensus.cc:2468] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 1.
I20251210 21:06:33.361299 28501 leader_election.cc:304] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:33.361418 28340 leader_election.cc:304] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters: 05b9cf62b62b446d8241939224cd3b22
I20251210 21:06:33.361586 28589 raft_consensus.cc:2804] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:06:33.361621 28575 raft_consensus.cc:2804] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:06:33.361665 28589 raft_consensus.cc:493] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:33.361677 28575 raft_consensus.cc:493] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:33.361729 28589 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.361733 28575 raft_consensus.cc:3060] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.364424 28575 raft_consensus.cc:515] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.364424 28589 raft_consensus.cc:515] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.365015 28575 leader_election.cc:290] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 election: Requested vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:33.365181 28589 leader_election.cc:290] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 2 election: Requested vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:33.365494 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22"
I20251210 21:06:33.365494 28461 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc72d6b1a4046f38bea511c85bfa3e3" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:33.365694 28541 raft_consensus.cc:3060] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.365711 28380 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c"
I20251210 21:06:33.365758 28461 raft_consensus.cc:3060] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.365841 28380 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.365815 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:33.365931 28460 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.368520 28461 raft_consensus.cc:2468] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 2.
I20251210 21:06:33.368583 28460 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 2.
I20251210 21:06:33.368517 28541 raft_consensus.cc:2410] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 09bf6175fa244f069e6dde0efb8eda2c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.368517 28380 raft_consensus.cc:2410] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 05b9cf62b62b446d8241939224cd3b22 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.368975 28340 leader_election.cc:304] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:33.369032 28501 leader_election.cc:304] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:33.369400 28589 raft_consensus.cc:2804] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:06:33.369678 28575 raft_consensus.cc:2804] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:06:33.369779 28575 raft_consensus.cc:697] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Becoming Leader. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Running, Role: LEADER
I20251210 21:06:33.370029 28575 consensus_queue.cc:237] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.370922 28589 raft_consensus.cc:697] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 2 LEADER]: Becoming Leader. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Running, Role: LEADER
I20251210 21:06:33.371125 28589 consensus_queue.cc:237] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.373088 27914 catalog_manager.cc:5654] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c reported cstate change: term changed from 1 to 2, leader changed from 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67) to 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65). New cstate: current_term: 2 leader_uuid: "09bf6175fa244f069e6dde0efb8eda2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:33.373240 27916 catalog_manager.cc:5654] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 reported cstate change: term changed from 1 to 2, leader changed from 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65) to 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67). New cstate: current_term: 2 leader_uuid: "05b9cf62b62b446d8241939224cd3b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:33.449577 28575 raft_consensus.cc:493] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:33.449776 28575 raft_consensus.cc:515] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.450407 28575 leader_election.cc:290] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:33.450965 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22" is_pre_election: true
I20251210 21:06:33.450963 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:33.451197 28460 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 1.
I20251210 21:06:33.451246 28541 raft_consensus.cc:2410] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09bf6175fa244f069e6dde0efb8eda2c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.451586 28340 leader_election.cc:304] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:33.451824 28575 raft_consensus.cc:2804] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251210 21:06:33.451917 28575 raft_consensus.cc:493] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:33.452018 28575 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.453934 28575 raft_consensus.cc:515] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.454496 28575 leader_election.cc:290] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 election: Requested vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:33.454885 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:33.454957 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22"
I20251210 21:06:33.455067 28460 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.455111 28541 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 1 FOLLOWER]: Advancing to term 2
I20251210 21:06:33.457131 28541 raft_consensus.cc:2410] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 09bf6175fa244f069e6dde0efb8eda2c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251210 21:06:33.457132 28460 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 2.
I20251210 21:06:33.457628 28340 leader_election.cc:304] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters: 05b9cf62b62b446d8241939224cd3b22
I20251210 21:06:33.457911 28575 raft_consensus.cc:2804] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Leader election won for term 2
I20251210 21:06:33.458058 28575 raft_consensus.cc:697] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Becoming Leader. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Running, Role: LEADER
I20251210 21:06:33.458360 28575 consensus_queue.cc:237] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.460793 27914 catalog_manager.cc:5654] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c reported cstate change: term changed from 1 to 2, leader changed from 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67) to 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65). New cstate: current_term: 2 leader_uuid: "09bf6175fa244f069e6dde0efb8eda2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:33.564754 28589 raft_consensus.cc:493] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:33.564981 28589 raft_consensus.cc:515] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.565668 28589 leader_election.cc:290] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:33.566213 28380 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c" is_pre_election: true
I20251210 21:06:33.566238 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:33.566499 28460 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 2.
I20251210 21:06:33.566834 28501 leader_election.cc:304] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters: 09bf6175fa244f069e6dde0efb8eda2c
I20251210 21:06:33.567114 28589 raft_consensus.cc:2804] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251210 21:06:33.567216 28589 raft_consensus.cc:493] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:33.567305 28589 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.569380 28589 raft_consensus.cc:515] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.570111 28589 leader_election.cc:290] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 3 election: Requested vote from peers 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101), febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:33.570531 28380 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09bf6175fa244f069e6dde0efb8eda2c"
I20251210 21:06:33.570531 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23f151792969458783a4b5eece7a02d5" candidate_uuid: "05b9cf62b62b446d8241939224cd3b22" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:33.570813 28460 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.572853 28460 raft_consensus.cc:2468] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b9cf62b62b446d8241939224cd3b22 in term 3.
I20251210 21:06:33.573233 28501 leader_election.cc:304] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b9cf62b62b446d8241939224cd3b22, febf21f9ae84480f8b4c3da2ae6c4706; no voters: 09bf6175fa244f069e6dde0efb8eda2c
I20251210 21:06:33.573519 28589 raft_consensus.cc:2804] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:06:33.573673 28589 raft_consensus.cc:697] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 3 LEADER]: Becoming Leader. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Running, Role: LEADER
I20251210 21:06:33.573920 28589 consensus_queue.cc:237] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.576429 27914 catalog_manager.cc:5654] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 reported cstate change: term changed from 2 to 3, leader changed from 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65) to 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67). New cstate: current_term: 3 leader_uuid: "05b9cf62b62b446d8241939224cd3b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:33.708978 28541 raft_consensus.cc:1240] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Rejecting Update request from peer 09bf6175fa244f069e6dde0efb8eda2c for earlier term 2. Current term is 3. Ops: []
I20251210 21:06:33.709717 28575 consensus_queue.cc:1059] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:33.710037 28578 raft_consensus.cc:3055] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:06:33.710143 28578 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Running, Role: LEADER
I20251210 21:06:33.710417 28578 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
I20251210 21:06:33.710683 28578 raft_consensus.cc:3060] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.747033 28578 raft_consensus.cc:493] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:33.747249 28578 raft_consensus.cc:515] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.747938 28578 leader_election.cc:290] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:33.748504 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" is_pre_election: true
I20251210 21:06:33.748517 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22" is_pre_election: true
I20251210 21:06:33.748721 28460 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 2.
I20251210 21:06:33.749110 28340 leader_election.cc:304] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters:
I20251210 21:06:33.749390 28578 raft_consensus.cc:2804] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251210 21:06:33.749505 28578 raft_consensus.cc:493] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:33.749595 28578 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.751703 28578 raft_consensus.cc:515] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.752256 28578 leader_election.cc:290] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 3 election: Requested vote from peers febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293), 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67:36489)
I20251210 21:06:33.752697 28541 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05b9cf62b62b446d8241939224cd3b22"
I20251210 21:06:33.752697 28460 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "257ae5d4e71d4a00859a8047e798516d" candidate_uuid: "09bf6175fa244f069e6dde0efb8eda2c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "febf21f9ae84480f8b4c3da2ae6c4706"
I20251210 21:06:33.752894 28460 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.754798 28460 raft_consensus.cc:2468] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6175fa244f069e6dde0efb8eda2c in term 3.
I20251210 21:06:33.755168 28340 leader_election.cc:304] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09bf6175fa244f069e6dde0efb8eda2c, febf21f9ae84480f8b4c3da2ae6c4706; no voters: 05b9cf62b62b446d8241939224cd3b22
I20251210 21:06:33.755501 28578 raft_consensus.cc:2804] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Leader election won for term 3
I20251210 21:06:33.755653 28578 raft_consensus.cc:697] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 3 LEADER]: Becoming Leader. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Running, Role: LEADER
I20251210 21:06:33.755893 28578 consensus_queue.cc:237] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.758461 27914 catalog_manager.cc:5654] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c reported cstate change: term changed from 2 to 3, leader changed from 05b9cf62b62b446d8241939224cd3b22 (127.21.127.67) to 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65). New cstate: current_term: 3 leader_uuid: "09bf6175fa244f069e6dde0efb8eda2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:33.804745 28380 raft_consensus.cc:1275] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Refusing update from remote peer febf21f9ae84480f8b4c3da2ae6c4706: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251210 21:06:33.805531 28584 consensus_queue.cc:1048] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:33.805825 28584 consensus_queue.cc:1243] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Peer 09bf6175fa244f069e6dde0efb8eda2c log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251210 21:06:33.807297 28380 pending_rounds.cc:85] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Aborting all ops after (but not including) 0
I20251210 21:06:33.807394 28380 pending_rounds.cc:107] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251210 21:06:33.807497 28380 raft_consensus.cc:2889] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251210 21:06:33.810827 28541 raft_consensus.cc:1275] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Refusing update from remote peer febf21f9ae84480f8b4c3da2ae6c4706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251210 21:06:33.811509 28600 consensus_queue.cc:1048] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:33.840830 28460 raft_consensus.cc:1240] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Rejecting Update request from peer 05b9cf62b62b446d8241939224cd3b22 for earlier term 2. Current term is 3. Ops: []
I20251210 21:06:33.841539 28589 consensus_queue.cc:1059] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:33.841815 28592 raft_consensus.cc:3055] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:06:33.841926 28592 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05b9cf62b62b446d8241939224cd3b22, State: Running, Role: LEADER
I20251210 21:06:33.842226 28592 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.842499 28592 raft_consensus.cc:3060] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Advancing to term 3
I20251210 21:06:33.853478 28460 raft_consensus.cc:1275] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251210 21:06:33.854063 28578 consensus_queue.cc:1048] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:33.859333 28541 raft_consensus.cc:1275] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20251210 21:06:33.859475 28541 pending_rounds.cc:85] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Aborting all ops after (but not including) 0
I20251210 21:06:33.859552 28541 pending_rounds.cc:107] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20251210 21:06:33.859644 28541 raft_consensus.cc:2889] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251210 21:06:33.860641 28575 consensus_queue.cc:1048] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251210 21:06:33.884845 28541 raft_consensus.cc:1240] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 3 LEADER]: Rejecting Update request from peer 09bf6175fa244f069e6dde0efb8eda2c for earlier term 2. Current term is 3. Ops: []
I20251210 21:06:33.885478 28575 consensus_queue.cc:1059] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:33.885735 28575 consensus_queue.cc:1243] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Peer 05b9cf62b62b446d8241939224cd3b22 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251210 21:06:33.885865 28578 raft_consensus.cc:3055] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Stepping down as leader of term 2
I20251210 21:06:33.885959 28578 raft_consensus.cc:740] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09bf6175fa244f069e6dde0efb8eda2c, State: Running, Role: LEADER
I20251210 21:06:33.886293 28578 consensus_queue.cc:260] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:33.886613 28578 raft_consensus.cc:3060] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Advancing to term 3
W20251210 21:06:33.915503 27964 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251210 21:06:33.917384 27964 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 257ae5d4e71d4a00859a8047e798516d, TS febf21f9ae84480f8b4c3da2ae6c4706 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
W20251210 21:06:33.917510 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 257ae5d4e71d4a00859a8047e798516d, TS febf21f9ae84480f8b4c3da2ae6c4706 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
W20251210 21:06:33.919329 27964 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 8244713e417e41c999b1a3d56357bb81, TS 09bf6175fa244f069e6dde0efb8eda2c -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
W20251210 21:06:33.919464 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 8244713e417e41c999b1a3d56357bb81, TS 09bf6175fa244f069e6dde0efb8eda2c -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } }
W20251210 21:06:33.921087 27964 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 2fc72d6b1a4046f38bea511c85bfa3e3, TS 05b9cf62b62b446d8241939224cd3b22 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
W20251210 21:06:33.921192 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 2fc72d6b1a4046f38bea511c85bfa3e3, TS 05b9cf62b62b446d8241939224cd3b22 -> TS fb19a7f9aad3441196a840e811c68e82 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } }
I20251210 21:06:34.031376 28380 raft_consensus.cc:1275] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Refusing update from remote peer 05b9cf62b62b446d8241939224cd3b22: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251210 21:06:34.032009 28592 consensus_queue.cc:1048] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:34.032321 28592 consensus_queue.cc:1243] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Peer 09bf6175fa244f069e6dde0efb8eda2c log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251210 21:06:34.033372 28380 pending_rounds.cc:85] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Aborting all ops after (but not including) 0
I20251210 21:06:34.033439 28380 pending_rounds.cc:107] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251210 21:06:34.033506 28380 raft_consensus.cc:2889] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251210 21:06:34.036335 28460 raft_consensus.cc:1275] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Refusing update from remote peer 05b9cf62b62b446d8241939224cd3b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251210 21:06:34.036917 28589 consensus_queue.cc:1048] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:34.214709 28460 raft_consensus.cc:1275] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251210 21:06:34.215521 28575 consensus_queue.cc:1048] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:34.220209 28541 raft_consensus.cc:1275] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251210 21:06:34.220988 28575 consensus_queue.cc:1048] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:34.221248 28575 consensus_queue.cc:1243] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Peer 05b9cf62b62b446d8241939224cd3b22 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251210 21:06:34.222244 28541 pending_rounds.cc:85] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Aborting all ops after (but not including) 0
I20251210 21:06:34.222332 28541 pending_rounds.cc:107] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251210 21:06:34.222404 28541 raft_consensus.cc:2889] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251210 21:06:34.925243 28460 consensus_queue.cc:237] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:34.927724 28541 raft_consensus.cc:1275] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Refusing update from remote peer febf21f9ae84480f8b4c3da2ae6c4706: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251210 21:06:34.928341 28600 consensus_queue.cc:1048] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:34.934085 28380 raft_consensus.cc:1275] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Refusing update from remote peer febf21f9ae84480f8b4c3da2ae6c4706: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251210 21:06:34.934650 28600 consensus_queue.cc:1048] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:34.935518 28600 raft_consensus.cc:2955] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.941845 27915 catalog_manager.cc:5654] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 reported cstate change: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New cstate: current_term: 3 leader_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:34.943045 28379 consensus_queue.cc:237] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
W20251210 21:06:34.944152 28422 consensus_peers.cc:597] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 -> Peer fb19a7f9aad3441196a840e811c68e82 (127.21.127.68:43571): Couldn't send request to peer fb19a7f9aad3441196a840e811c68e82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8244713e417e41c999b1a3d56357bb81. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:34.949811 28541 raft_consensus.cc:2955] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.954890 28380 raft_consensus.cc:2955] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.956451 28460 raft_consensus.cc:1275] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:06:34.957139 28575 consensus_queue.cc:1048] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:06:34.960100 28541 raft_consensus.cc:1275] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251210 21:06:34.960702 28575 consensus_queue.cc:1048] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:06:34.961551 28575 raft_consensus.cc:2955] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.962850 28460 raft_consensus.cc:2955] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.966180 28541 raft_consensus.cc:2955] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.966701 27914 catalog_manager.cc:5654] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c reported cstate change: config changed from index -1 to 2, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New cstate: current_term: 2 leader_uuid: "09bf6175fa244f069e6dde0efb8eda2c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:34.969074 28379 consensus_queue.cc:237] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:34.971350 28460 raft_consensus.cc:1275] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251210 21:06:34.971519 28540 raft_consensus.cc:1275] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Refusing update from remote peer 09bf6175fa244f069e6dde0efb8eda2c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251210 21:06:34.971939 28575 consensus_queue.cc:1048] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:34.972280 28605 consensus_queue.cc:1048] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251210 21:06:34.973001 28341 consensus_peers.cc:597] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c -> Peer fb19a7f9aad3441196a840e811c68e82 (127.21.127.68:43571): Couldn't send request to peer fb19a7f9aad3441196a840e811c68e82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 257ae5d4e71d4a00859a8047e798516d. This is attempt 1: this message will repeat every 5th retry.
W20251210 21:06:34.973263 28341 consensus_peers.cc:597] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c -> Peer fb19a7f9aad3441196a840e811c68e82 (127.21.127.68:43571): Couldn't send request to peer fb19a7f9aad3441196a840e811c68e82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fc72d6b1a4046f38bea511c85bfa3e3. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:34.974758 28605 raft_consensus.cc:2955] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.975353 28540 raft_consensus.cc:2955] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.976295 28460 raft_consensus.cc:2955] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } } }
I20251210 21:06:34.979133 27914 catalog_manager.cc:5654] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c reported cstate change: config changed from index -1 to 3, NON_VOTER fb19a7f9aad3441196a840e811c68e82 (127.21.127.68) added. New cstate: current_term: 3 leader_uuid: "09bf6175fa244f069e6dde0efb8eda2c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:35.452430 28627 ts_tablet_manager.cc:933] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: Initiating tablet copy from peer febf21f9ae84480f8b4c3da2ae6c4706 (127.21.127.66:41293)
I20251210 21:06:35.453187 28627 tablet_copy_client.cc:323] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:41293
I20251210 21:06:35.459968 28631 ts_tablet_manager.cc:933] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: Initiating tablet copy from peer 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:35.460616 28631 tablet_copy_client.cc:323] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:39101
I20251210 21:06:35.464562 28472 tablet_copy_service.cc:140] P febf21f9ae84480f8b4c3da2ae6c4706: Received BeginTabletCopySession request for tablet 8244713e417e41c999b1a3d56357bb81 from peer fb19a7f9aad3441196a840e811c68e82 ({username='slave'} at 127.0.0.1:51794)
I20251210 21:06:35.464758 28472 tablet_copy_service.cc:161] P febf21f9ae84480f8b4c3da2ae6c4706: Beginning new tablet copy session on tablet 8244713e417e41c999b1a3d56357bb81 from peer fb19a7f9aad3441196a840e811c68e82 at {username='slave'} at 127.0.0.1:51794: session id = fb19a7f9aad3441196a840e811c68e82-8244713e417e41c999b1a3d56357bb81
I20251210 21:06:35.467918 28472 tablet_copy_source_session.cc:215] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:35.468840 28391 tablet_copy_service.cc:140] P 09bf6175fa244f069e6dde0efb8eda2c: Received BeginTabletCopySession request for tablet 257ae5d4e71d4a00859a8047e798516d from peer fb19a7f9aad3441196a840e811c68e82 ({username='slave'} at 127.0.0.1:33224)
I20251210 21:06:35.469164 28391 tablet_copy_service.cc:161] P 09bf6175fa244f069e6dde0efb8eda2c: Beginning new tablet copy session on tablet 257ae5d4e71d4a00859a8047e798516d from peer fb19a7f9aad3441196a840e811c68e82 at {username='slave'} at 127.0.0.1:33224: session id = fb19a7f9aad3441196a840e811c68e82-257ae5d4e71d4a00859a8047e798516d
I20251210 21:06:35.472128 28391 tablet_copy_source_session.cc:215] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:35.472992 28627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8244713e417e41c999b1a3d56357bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:35.473489 28631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257ae5d4e71d4a00859a8047e798516d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:35.478854 28631 tablet_copy_client.cc:806] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:35.479068 28627 tablet_copy_client.cc:806] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:35.479327 28631 tablet_copy_client.cc:670] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:35.479434 28627 tablet_copy_client.cc:670] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:35.481076 28627 tablet_copy_client.cc:538] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:35.481176 28631 tablet_copy_client.cc:538] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:35.484810 28627 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: Bootstrap starting.
I20251210 21:06:35.484884 28631 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: Bootstrap starting.
I20251210 21:06:35.491686 28631 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:35.492079 28631 tablet_bootstrap.cc:492] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: Bootstrap complete.
I20251210 21:06:35.492400 28631 ts_tablet_manager.cc:1403] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20251210 21:06:35.493626 28631 raft_consensus.cc:359] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:35.493868 28631 raft_consensus.cc:740] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: fb19a7f9aad3441196a840e811c68e82, State: Initialized, Role: LEARNER
I20251210 21:06:35.494132 28631 consensus_queue.cc:260] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } attrs { replace: true } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:35.494943 28631 ts_tablet_manager.cc:1434] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:35.495929 28391 tablet_copy_service.cc:342] P 09bf6175fa244f069e6dde0efb8eda2c: Request end of tablet copy session fb19a7f9aad3441196a840e811c68e82-257ae5d4e71d4a00859a8047e798516d received from {username='slave'} at 127.0.0.1:33224
I20251210 21:06:35.496066 28391 tablet_copy_service.cc:434] P 09bf6175fa244f069e6dde0efb8eda2c: ending tablet copy session fb19a7f9aad3441196a840e811c68e82-257ae5d4e71d4a00859a8047e798516d on tablet 257ae5d4e71d4a00859a8047e798516d with peer fb19a7f9aad3441196a840e811c68e82
I20251210 21:06:35.497828 28627 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:35.498267 28627 tablet_bootstrap.cc:492] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: Bootstrap complete.
I20251210 21:06:35.498539 28627 ts_tablet_manager.cc:1403] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20251210 21:06:35.499372 28627 raft_consensus.cc:359] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:35.499673 28627 raft_consensus.cc:740] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: fb19a7f9aad3441196a840e811c68e82, State: Initialized, Role: LEARNER
I20251210 21:06:35.499863 28627 consensus_queue.cc:260] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } attrs { replace: true } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:35.500433 28627 ts_tablet_manager.cc:1434] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20251210 21:06:35.501055 28472 tablet_copy_service.cc:342] P febf21f9ae84480f8b4c3da2ae6c4706: Request end of tablet copy session fb19a7f9aad3441196a840e811c68e82-8244713e417e41c999b1a3d56357bb81 received from {username='slave'} at 127.0.0.1:51794
I20251210 21:06:35.501188 28472 tablet_copy_service.cc:434] P febf21f9ae84480f8b4c3da2ae6c4706: ending tablet copy session fb19a7f9aad3441196a840e811c68e82-8244713e417e41c999b1a3d56357bb81 on tablet 8244713e417e41c999b1a3d56357bb81 with peer fb19a7f9aad3441196a840e811c68e82
I20251210 21:06:35.509506 28627 ts_tablet_manager.cc:933] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: Initiating tablet copy from peer 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101)
I20251210 21:06:35.510109 28627 tablet_copy_client.cc:323] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.65:39101
I20251210 21:06:35.510746 28391 tablet_copy_service.cc:140] P 09bf6175fa244f069e6dde0efb8eda2c: Received BeginTabletCopySession request for tablet 2fc72d6b1a4046f38bea511c85bfa3e3 from peer fb19a7f9aad3441196a840e811c68e82 ({username='slave'} at 127.0.0.1:33224)
I20251210 21:06:35.510877 28391 tablet_copy_service.cc:161] P 09bf6175fa244f069e6dde0efb8eda2c: Beginning new tablet copy session on tablet 2fc72d6b1a4046f38bea511c85bfa3e3 from peer fb19a7f9aad3441196a840e811c68e82 at {username='slave'} at 127.0.0.1:33224: session id = fb19a7f9aad3441196a840e811c68e82-2fc72d6b1a4046f38bea511c85bfa3e3
I20251210 21:06:35.513185 28391 tablet_copy_source_session.cc:215] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:35.514168 28627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc72d6b1a4046f38bea511c85bfa3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:35.519574 28627 tablet_copy_client.cc:806] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:35.519848 28627 tablet_copy_client.cc:670] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:35.521263 28627 tablet_copy_client.cc:538] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:35.524322 28627 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: Bootstrap starting.
I20251210 21:06:35.531075 28627 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:35.531548 28627 tablet_bootstrap.cc:492] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: Bootstrap complete.
I20251210 21:06:35.531800 28627 ts_tablet_manager.cc:1403] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20251210 21:06:35.532593 28627 raft_consensus.cc:359] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:35.532783 28627 raft_consensus.cc:740] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: fb19a7f9aad3441196a840e811c68e82, State: Initialized, Role: LEARNER
I20251210 21:06:35.533006 28627 consensus_queue.cc:260] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09bf6175fa244f069e6dde0efb8eda2c" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 39101 } } peers { permanent_uuid: "febf21f9ae84480f8b4c3da2ae6c4706" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 41293 } attrs { replace: true } } peers { permanent_uuid: "05b9cf62b62b446d8241939224cd3b22" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 36489 } } peers { permanent_uuid: "fb19a7f9aad3441196a840e811c68e82" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 43571 } attrs { promote: true } }
I20251210 21:06:35.533610 28627 ts_tablet_manager.cc:1434] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:35.534363 28391 tablet_copy_service.cc:342] P 09bf6175fa244f069e6dde0efb8eda2c: Request end of tablet copy session fb19a7f9aad3441196a840e811c68e82-2fc72d6b1a4046f38bea511c85bfa3e3 received from {username='slave'} at 127.0.0.1:33224
I20251210 21:06:35.534484 28391 tablet_copy_service.cc:434] P 09bf6175fa244f069e6dde0efb8eda2c: ending tablet copy session fb19a7f9aad3441196a840e811c68e82-2fc72d6b1a4046f38bea511c85bfa3e3 on tablet 2fc72d6b1a4046f38bea511c85bfa3e3 with peer fb19a7f9aad3441196a840e811c68e82
I20251210 21:06:35.985510 28290 raft_consensus.cc:1217] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251210 21:06:36.009630 28290 raft_consensus.cc:1217] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20251210 21:06:36.031137 28290 raft_consensus.cc:1217] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251210 21:06:36.238334 22013 tablet_server.cc:178] TabletServer@127.21.127.65:39101 shutting down...
W20251210 21:06:36.240070 27963 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20251210 21:06:36.242537 27964 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:06:36.242650 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:06:36.246814 27964 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:06:36.246992 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251210 21:06:36.247957 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:36.248179 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.248543 22013 raft_consensus.cc:2243] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.248790 22013 raft_consensus.cc:2272] T 8244713e417e41c999b1a3d56357bb81 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.249557 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.249675 22013 raft_consensus.cc:2243] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 LEADER]: Raft consensus shutting down.
I20251210 21:06:36.250042 22013 raft_consensus.cc:2272] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 09bf6175fa244f069e6dde0efb8eda2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.250662 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.250787 22013 raft_consensus.cc:2243] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 3 LEADER]: Raft consensus shutting down.
I20251210 21:06:36.251122 22013 raft_consensus.cc:2272] T 257ae5d4e71d4a00859a8047e798516d P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.251715 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.251874 22013 raft_consensus.cc:2243] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.252014 22013 raft_consensus.cc:2272] T 23f151792969458783a4b5eece7a02d5 P 09bf6175fa244f069e6dde0efb8eda2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.258772 22013 tablet_server.cc:195] TabletServer@127.21.127.65:39101 shutdown complete.
I20251210 21:06:36.262876 22013 tablet_server.cc:178] TabletServer@127.21.127.66:41293 shutting down...
W20251210 21:06:36.265553 27964 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:06:36.265905 27964 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251210 21:06:36.272054 28503 consensus_peers.cc:597] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 -> Peer 09bf6175fa244f069e6dde0efb8eda2c (127.21.127.65:39101): Couldn't send request to peer 09bf6175fa244f069e6dde0efb8eda2c. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:39101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:36.273054 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:36.284782 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.285003 22013 raft_consensus.cc:2243] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 LEADER]: Raft consensus shutting down.
I20251210 21:06:36.285399 22013 raft_consensus.cc:2272] T 8244713e417e41c999b1a3d56357bb81 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.286021 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.286141 22013 raft_consensus.cc:2243] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.286290 22013 raft_consensus.cc:2272] T 2fc72d6b1a4046f38bea511c85bfa3e3 P febf21f9ae84480f8b4c3da2ae6c4706 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.286710 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.286805 22013 raft_consensus.cc:2243] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.286923 22013 raft_consensus.cc:2272] T 257ae5d4e71d4a00859a8047e798516d P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.287297 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.287388 22013 raft_consensus.cc:2243] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.287523 22013 raft_consensus.cc:2272] T 23f151792969458783a4b5eece7a02d5 P febf21f9ae84480f8b4c3da2ae6c4706 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.292058 22013 tablet_server.cc:195] TabletServer@127.21.127.66:41293 shutdown complete.
I20251210 21:06:36.295707 22013 tablet_server.cc:178] TabletServer@127.21.127.67:36489 shutting down...
I20251210 21:06:36.305243 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:36.305548 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.305721 22013 raft_consensus.cc:2243] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.305929 22013 raft_consensus.cc:2272] T 8244713e417e41c999b1a3d56357bb81 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.306547 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.306679 22013 raft_consensus.cc:2243] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.306841 22013 raft_consensus.cc:2272] T 2fc72d6b1a4046f38bea511c85bfa3e3 P 05b9cf62b62b446d8241939224cd3b22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.307278 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.307391 22013 raft_consensus.cc:2243] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:36.307571 22013 raft_consensus.cc:2272] T 257ae5d4e71d4a00859a8047e798516d P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.307962 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:36.308079 22013 raft_consensus.cc:2243] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 3 LEADER]: Raft consensus shutting down.
I20251210 21:06:36.308342 22013 raft_consensus.cc:2272] T 23f151792969458783a4b5eece7a02d5 P 05b9cf62b62b446d8241939224cd3b22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.323043 22013 tablet_server.cc:195] TabletServer@127.21.127.67:36489 shutdown complete.
I20251210 21:06:36.326889 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:06:36.335577 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:36.335834 22013 tablet_replica.cc:333] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82: stopping tablet replica
I20251210 21:06:36.336016 22013 raft_consensus.cc:2243] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82 [term 2 LEARNER]: Raft consensus shutting down.
I20251210 21:06:36.336184 22013 raft_consensus.cc:2272] T 2fc72d6b1a4046f38bea511c85bfa3e3 P fb19a7f9aad3441196a840e811c68e82 [term 2 LEARNER]: Raft consensus is shut down!
I20251210 21:06:36.336779 22013 tablet_replica.cc:333] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82: stopping tablet replica
I20251210 21:06:36.336915 22013 raft_consensus.cc:2243] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Raft consensus shutting down.
I20251210 21:06:36.337044 22013 raft_consensus.cc:2272] T 257ae5d4e71d4a00859a8047e798516d P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Raft consensus is shut down!
I20251210 21:06:36.337497 22013 tablet_replica.cc:333] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82: stopping tablet replica
I20251210 21:06:36.337618 22013 raft_consensus.cc:2243] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Raft consensus shutting down.
I20251210 21:06:36.337735 22013 raft_consensus.cc:2272] T 8244713e417e41c999b1a3d56357bb81 P fb19a7f9aad3441196a840e811c68e82 [term 3 LEARNER]: Raft consensus is shut down!
I20251210 21:06:36.351395 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:06:36.354930 22013 master.cc:561] Master@127.21.127.126:37443 shutting down...
I20251210 21:06:36.362469 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:36.362713 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:36.362834 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P e86e248d51034501a7e71e2d1f3cea8b: stopping tablet replica
I20251210 21:06:36.376111 22013 master.cc:583] Master@127.21.127.126:37443 shutdown complete.
[ OK ] AutoRebalancerTest.TestHandlingFailedTservers (11605 ms)
[ RUN ] AutoRebalancerTest.TestDeletedTables
I20251210 21:06:36.389703 22013 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.127.126:44701
I20251210 21:06:36.390220 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:36.392459 28649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.392539 28650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.392927 28652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:36.392935 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:36.393298 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:36.393352 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:36.393381 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400796393377 us; error 0 us; skew 500 ppm
I20251210 21:06:36.393517 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:36.394672 22013 webserver.cc:492] Webserver started at http://127.21.127.126:32953/ using document root <none> and password file <none>
I20251210 21:06:36.394871 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:36.394929 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:36.395062 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:36.395751 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/master-0-root/instance:
uuid: "96baabc14c4f4e92b2892043f191b224"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.398110 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:36.399600 28657 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.399922 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:36.400029 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/master-0-root
uuid: "96baabc14c4f4e92b2892043f191b224"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.400132 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:36.429376 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:36.429916 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:36.449716 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.126:44701
I20251210 21:06:36.449725 28708 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.126:44701 every 8 connection(s)
I20251210 21:06:36.451048 28709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:36.455207 28709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: Bootstrap starting.
I20251210 21:06:36.456391 28709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:36.457814 28709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: No bootstrap required, opened a new log
I20251210 21:06:36.458402 28709 raft_consensus.cc:359] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96baabc14c4f4e92b2892043f191b224" member_type: VOTER }
I20251210 21:06:36.458524 28709 raft_consensus.cc:385] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:36.458559 28709 raft_consensus.cc:740] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96baabc14c4f4e92b2892043f191b224, State: Initialized, Role: FOLLOWER
I20251210 21:06:36.458743 28709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96baabc14c4f4e92b2892043f191b224" member_type: VOTER }
I20251210 21:06:36.458844 28709 raft_consensus.cc:399] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251210 21:06:36.458894 28709 raft_consensus.cc:493] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251210 21:06:36.458957 28709 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:36.460764 28709 raft_consensus.cc:515] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96baabc14c4f4e92b2892043f191b224" member_type: VOTER }
I20251210 21:06:36.460945 28709 leader_election.cc:304] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96baabc14c4f4e92b2892043f191b224; no voters:
I20251210 21:06:36.461355 28709 leader_election.cc:290] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251210 21:06:36.461481 28712 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:36.461905 28712 raft_consensus.cc:697] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 1 LEADER]: Becoming Leader. State: Replica: 96baabc14c4f4e92b2892043f191b224, State: Running, Role: LEADER
I20251210 21:06:36.462123 28709 sys_catalog.cc:565] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [sys.catalog]: configured and running, proceeding with master startup.
I20251210 21:06:36.462191 28712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96baabc14c4f4e92b2892043f191b224" member_type: VOTER }
I20251210 21:06:36.463197 28714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96baabc14c4f4e92b2892043f191b224. Latest consensus state: current_term: 1 leader_uuid: "96baabc14c4f4e92b2892043f191b224" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96baabc14c4f4e92b2892043f191b224" member_type: VOTER } }
I20251210 21:06:36.463354 28714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [sys.catalog]: This master's current role is: LEADER
I20251210 21:06:36.465606 28713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96baabc14c4f4e92b2892043f191b224" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96baabc14c4f4e92b2892043f191b224" member_type: VOTER } }
I20251210 21:06:36.465755 28713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [sys.catalog]: This master's current role is: LEADER
I20251210 21:06:36.466122 28719 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251210 21:06:36.467882 28719 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251210 21:06:36.468246 22013 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251210 21:06:36.470641 28719 catalog_manager.cc:1357] Generated new cluster ID: 7624127466344c21969b0c6036c2e703
I20251210 21:06:36.470710 28719 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251210 21:06:36.480304 28719 catalog_manager.cc:1380] Generated new certificate authority record
I20251210 21:06:36.481122 28719 catalog_manager.cc:1514] Loading token signing keys...
I20251210 21:06:36.491683 28719 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: Generated new TSK 0
I20251210 21:06:36.491946 28719 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251210 21:06:36.501137 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:36.503798 28731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.503795 28730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.504385 28733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:36.504892 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:36.505112 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:36.505162 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:36.505190 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400796505186 us; error 0 us; skew 500 ppm
I20251210 21:06:36.505309 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:36.506407 22013 webserver.cc:492] Webserver started at http://127.21.127.65:41797/ using document root <none> and password file <none>
I20251210 21:06:36.506599 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:36.506649 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:36.506749 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:36.507344 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/instance:
uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.509827 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:36.511298 28738 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.511700 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:36.511812 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.511914 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:36.521595 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:36.522104 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:36.522621 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:36.523522 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:36.523578 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.523663 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:36.523705 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.535679 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:43855
I20251210 21:06:36.535691 28800 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:43855 every 8 connection(s)
I20251210 21:06:36.537269 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:36.539814 28806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.539831 28805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.541776 28808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:36.541818 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:36.542128 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:36.542205 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:36.542244 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400796542240 us; error 0 us; skew 500 ppm
I20251210 21:06:36.542382 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:36.543655 22013 webserver.cc:492] Webserver started at http://127.21.127.66:44017/ using document root <none> and password file <none>
I20251210 21:06:36.543857 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:36.543913 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:36.544018 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:36.544721 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root/instance:
uuid: "00cefb4b862b4514af558a61c220754f"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.546121 28801 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44701
I20251210 21:06:36.546252 28801 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:36.546593 28801 heartbeater.cc:507] Master 127.21.127.126:44701 requested a full tablet report, sending...
I20251210 21:06:36.547380 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20251210 21:06:36.547559 28674 ts_manager.cc:194] Registered new tserver with Master: 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855)
I20251210 21:06:36.548604 28674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60462
I20251210 21:06:36.549117 28813 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.549449 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:36.549531 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root
uuid: "00cefb4b862b4514af558a61c220754f"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.549635 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:36.576061 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:36.576645 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:36.577207 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:36.578047 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:36.578107 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.578173 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:36.578207 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.590322 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.66:33303
I20251210 21:06:36.590348 28875 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.66:33303 every 8 connection(s)
I20251210 21:06:36.591845 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:36.594691 28879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.595235 28880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:36.595248 28882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:36.596000 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:36.596252 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:36.596333 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:36.596426 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400796596420 us; error 0 us; skew 500 ppm
I20251210 21:06:36.596580 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:36.597823 22013 webserver.cc:492] Webserver started at http://127.21.127.67:38291/ using document root <none> and password file <none>
I20251210 21:06:36.598062 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:36.598143 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:36.598263 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:36.598390 28876 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44701
I20251210 21:06:36.598515 28876 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:36.598839 28876 heartbeater.cc:507] Master 127.21.127.126:44701 requested a full tablet report, sending...
I20251210 21:06:36.598891 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root/instance:
uuid: "ed0eda03e402411db17cfa89ad03980b"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.599797 28674 ts_manager.cc:194] Registered new tserver with Master: 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:36.600827 28674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60478
I20251210 21:06:36.602043 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251210 21:06:36.603468 28887 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.603828 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:36.603909 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root
uuid: "ed0eda03e402411db17cfa89ad03980b"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:36.604032 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:36.620469 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:36.620950 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:36.621440 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:36.622246 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:36.622303 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.622354 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:36.622391 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:36.632753 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.67:41227
I20251210 21:06:36.632812 28949 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.67:41227 every 8 connection(s)
I20251210 21:06:36.639405 28950 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44701
I20251210 21:06:36.639580 28950 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:36.639910 28950 heartbeater.cc:507] Master 127.21.127.126:44701 requested a full tablet report, sending...
I20251210 21:06:36.640815 28674 ts_manager.cc:194] Registered new tserver with Master: ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:36.641568 22013 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007908461s
I20251210 21:06:36.641604 28674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60484
I20251210 21:06:37.550135 28801 heartbeater.cc:499] Master 127.21.127.126:44701 was elected leader, sending a full tablet report...
I20251210 21:06:37.602155 28876 heartbeater.cc:499] Master 127.21.127.126:44701 was elected leader, sending a full tablet report...
I20251210 21:06:37.643124 28950 heartbeater.cc:499] Master 127.21.127.126:44701 was elected leader, sending a full tablet report...
I20251210 21:06:37.669284 22013 test_util.cc:276] Using random seed: 1735288732
I20251210 21:06:37.685472 28674 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60500:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:06:37.686208 28674 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:06:37.706489 28766 tablet_service.cc:1505] Processing CreateTablet for tablet 15715890f89e4598805c5cc2ff489db3 (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:37.706626 28765 tablet_service.cc:1505] Processing CreateTablet for tablet 18db8ba4df094f5f8e27ae62a446c78a (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:37.707041 28765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18db8ba4df094f5f8e27ae62a446c78a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.709169 28766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15715890f89e4598805c5cc2ff489db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.713192 28841 tablet_service.cc:1505] Processing CreateTablet for tablet 15715890f89e4598805c5cc2ff489db3 (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:37.713192 28840 tablet_service.cc:1505] Processing CreateTablet for tablet 18db8ba4df094f5f8e27ae62a446c78a (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:37.713477 28839 tablet_service.cc:1505] Processing CreateTablet for tablet 8dcdcf8eac50460b83c74ee9a90520e7 (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:37.713680 28841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15715890f89e4598805c5cc2ff489db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.713841 28840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18db8ba4df094f5f8e27ae62a446c78a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.714051 28839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dcdcf8eac50460b83c74ee9a90520e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.714331 28838 tablet_service.cc:1505] Processing CreateTablet for tablet 94d66b996d8742b192d3ab2b22decb5d (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:37.714687 28838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94d66b996d8742b192d3ab2b22decb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.717377 28915 tablet_service.cc:1505] Processing CreateTablet for tablet 15715890f89e4598805c5cc2ff489db3 (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:37.717728 28915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15715890f89e4598805c5cc2ff489db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.720342 28764 tablet_service.cc:1505] Processing CreateTablet for tablet 8dcdcf8eac50460b83c74ee9a90520e7 (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:37.720559 28763 tablet_service.cc:1505] Processing CreateTablet for tablet 94d66b996d8742b192d3ab2b22decb5d (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:37.720723 28764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dcdcf8eac50460b83c74ee9a90520e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.720891 28763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94d66b996d8742b192d3ab2b22decb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.720890 28971 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:37.722469 28971 tablet_bootstrap.cc:654] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.724277 28971 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:37.724493 28971 ts_tablet_manager.cc:1403] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:37.725322 28971 raft_consensus.cc:359] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.725625 28971 raft_consensus.cc:385] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.725723 28971 raft_consensus.cc:740] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.725983 28971 consensus_queue.cc:260] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.726800 28971 ts_tablet_manager.cc:1434] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:37.727145 28971 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:37.728636 28971 tablet_bootstrap.cc:654] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.730168 28914 tablet_service.cc:1505] Processing CreateTablet for tablet 18db8ba4df094f5f8e27ae62a446c78a (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:37.730593 28914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18db8ba4df094f5f8e27ae62a446c78a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.737438 28975 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:37.737514 28913 tablet_service.cc:1505] Processing CreateTablet for tablet 94d66b996d8742b192d3ab2b22decb5d (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:37.737990 28913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94d66b996d8742b192d3ab2b22decb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.739284 28975 tablet_bootstrap.cc:654] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.740675 28970 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:37.742303 28970 tablet_bootstrap.cc:654] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.743460 28914 tablet_service.cc:1505] Processing CreateTablet for tablet 8dcdcf8eac50460b83c74ee9a90520e7 (DEFAULT_TABLE table=test-workload [id=e2e260b5487e4123bab2b5c51b620238]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:37.743870 28914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dcdcf8eac50460b83c74ee9a90520e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:37.744102 28975 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:37.744318 28975 ts_tablet_manager.cc:1403] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20251210 21:06:37.745204 28971 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:37.745129 28975 raft_consensus.cc:359] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.745335 28971 ts_tablet_manager.cc:1403] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.005s
I20251210 21:06:37.745455 28975 raft_consensus.cc:385] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.745575 28975 raft_consensus.cc:740] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.745844 28970 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:37.745878 28975 consensus_queue.cc:260] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.746011 28970 ts_tablet_manager.cc:1403] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251210 21:06:37.746147 28971 raft_consensus.cc:359] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.746326 28971 raft_consensus.cc:385] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.746428 28971 raft_consensus.cc:740] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.746702 28971 consensus_queue.cc:260] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.746829 28970 raft_consensus.cc:359] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.747030 28970 raft_consensus.cc:385] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.747125 28970 raft_consensus.cc:740] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.747215 28971 ts_tablet_manager.cc:1434] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:37.748266 28971 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:37.747399 28970 consensus_queue.cc:260] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.750394 28970 ts_tablet_manager.cc:1434] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251210 21:06:37.750675 28970 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:37.750897 28971 tablet_bootstrap.cc:654] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.752143 28970 tablet_bootstrap.cc:654] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.753240 28975 ts_tablet_manager.cc:1434] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.009s user 0.000s sys 0.008s
I20251210 21:06:37.755998 28970 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:37.756175 28970 ts_tablet_manager.cc:1403] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20251210 21:06:37.756204 28975 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:37.757002 28970 raft_consensus.cc:359] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.757216 28970 raft_consensus.cc:385] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.757268 28971 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:37.757289 28970 raft_consensus.cc:740] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.757452 28971 ts_tablet_manager.cc:1403] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20251210 21:06:37.757563 28970 consensus_queue.cc:260] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.758150 28970 ts_tablet_manager.cc:1434] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:37.758205 28971 raft_consensus.cc:359] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.758307 28975 tablet_bootstrap.cc:654] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.758404 28971 raft_consensus.cc:385] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.758417 28970 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:37.758526 28971 raft_consensus.cc:740] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.758764 28971 consensus_queue.cc:260] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.759989 28971 ts_tablet_manager.cc:1434] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:37.760093 28970 tablet_bootstrap.cc:654] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.760284 28971 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:37.761667 28971 tablet_bootstrap.cc:654] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.767530 28970 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:37.767736 28970 ts_tablet_manager.cc:1403] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20251210 21:06:37.767760 28971 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:37.767979 28971 ts_tablet_manager.cc:1403] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20251210 21:06:37.768086 28975 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:37.768211 28975 ts_tablet_manager.cc:1403] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20251210 21:06:37.768608 28971 raft_consensus.cc:359] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.768635 28970 raft_consensus.cc:359] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.768754 28971 raft_consensus.cc:385] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.768788 28970 raft_consensus.cc:385] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.768828 28971 raft_consensus.cc:740] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.768847 28975 raft_consensus.cc:359] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.768900 28970 raft_consensus.cc:740] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.768998 28975 raft_consensus.cc:385] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.769053 28975 raft_consensus.cc:740] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.769152 28971 consensus_queue.cc:260] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.769202 28975 consensus_queue.cc:260] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.769150 28970 consensus_queue.cc:260] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.769712 28971 ts_tablet_manager.cc:1434] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:37.769797 28970 ts_tablet_manager.cc:1434] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:37.769812 28975 ts_tablet_manager.cc:1434] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:06:37.770092 28975 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:37.770092 28970 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:37.771659 28975 tablet_bootstrap.cc:654] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.771659 28970 tablet_bootstrap.cc:654] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.776407 28970 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:37.776407 28975 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:37.776584 28975 ts_tablet_manager.cc:1403] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251210 21:06:37.776584 28970 ts_tablet_manager.cc:1403] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20251210 21:06:37.777573 28970 raft_consensus.cc:359] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.777573 28975 raft_consensus.cc:359] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.777787 28970 raft_consensus.cc:385] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.777887 28970 raft_consensus.cc:740] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.778101 28970 consensus_queue.cc:260] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.778478 28970 ts_tablet_manager.cc:1434] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:37.777794 28975 raft_consensus.cc:385] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.778659 28975 raft_consensus.cc:740] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.778853 28975 consensus_queue.cc:260] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.779265 28975 ts_tablet_manager.cc:1434] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251210 21:06:37.779561 28975 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:37.780891 28975 tablet_bootstrap.cc:654] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:37.782709 28975 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:37.782850 28975 ts_tablet_manager.cc:1403] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251210 21:06:37.783428 28975 raft_consensus.cc:359] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.783596 28975 raft_consensus.cc:385] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:37.783638 28975 raft_consensus.cc:740] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:37.783787 28975 consensus_queue.cc:260] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.784157 28975 ts_tablet_manager.cc:1434] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:37.839248 28978 raft_consensus.cc:493] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:37.839432 28978 raft_consensus.cc:515] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.840318 28978 leader_election.cc:290] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:37.846411 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8dcdcf8eac50460b83c74ee9a90520e7" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" is_pre_election: true
I20251210 21:06:37.846414 28851 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8dcdcf8eac50460b83c74ee9a90520e7" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f" is_pre_election: true
I20251210 21:06:37.846635 28776 raft_consensus.cc:2468] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 0.
I20251210 21:06:37.846666 28851 raft_consensus.cc:2468] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 0.
I20251210 21:06:37.847039 28889 leader_election.cc:304] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:37.847344 28978 raft_consensus.cc:2804] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:37.847424 28978 raft_consensus.cc:493] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:37.847520 28978 raft_consensus.cc:3060] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:37.849485 28978 raft_consensus.cc:515] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.850194 28978 leader_election.cc:290] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 election: Requested vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:37.850646 28851 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8dcdcf8eac50460b83c74ee9a90520e7" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f"
I20251210 21:06:37.850684 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8dcdcf8eac50460b83c74ee9a90520e7" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
I20251210 21:06:37.850793 28851 raft_consensus.cc:3060] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:37.850865 28776 raft_consensus.cc:3060] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:37.852854 28851 raft_consensus.cc:2468] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 1.
I20251210 21:06:37.852849 28776 raft_consensus.cc:2468] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 1.
I20251210 21:06:37.853274 28891 leader_election.cc:304] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:37.853545 28978 raft_consensus.cc:2804] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:37.853924 28978 raft_consensus.cc:697] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Becoming Leader. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Running, Role: LEADER
I20251210 21:06:37.854178 28978 consensus_queue.cc:237] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.856782 28674 catalog_manager.cc:5654] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b reported cstate change: term changed from 0 to 1, leader changed from <none> to ed0eda03e402411db17cfa89ad03980b (127.21.127.67). New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:37.898133 28978 raft_consensus.cc:493] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:37.898360 28978 raft_consensus.cc:515] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.898989 28978 leader_election.cc:290] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:37.899650 28851 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94d66b996d8742b192d3ab2b22decb5d" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f" is_pre_election: true
I20251210 21:06:37.899652 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94d66b996d8742b192d3ab2b22decb5d" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" is_pre_election: true
I20251210 21:06:37.899850 28851 raft_consensus.cc:2468] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 0.
I20251210 21:06:37.899876 28776 raft_consensus.cc:2468] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 0.
I20251210 21:06:37.900234 28891 leader_election.cc:304] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:37.900514 28978 raft_consensus.cc:2804] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:37.900616 28978 raft_consensus.cc:493] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:37.900688 28978 raft_consensus.cc:3060] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:37.902827 28978 raft_consensus.cc:515] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.903497 28978 leader_election.cc:290] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 election: Requested vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:37.904038 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94d66b996d8742b192d3ab2b22decb5d" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
I20251210 21:06:37.904086 28851 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94d66b996d8742b192d3ab2b22decb5d" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f"
I20251210 21:06:37.904238 28851 raft_consensus.cc:3060] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:37.904222 28776 raft_consensus.cc:3060] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:37.906399 28851 raft_consensus.cc:2468] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 1.
I20251210 21:06:37.906399 28776 raft_consensus.cc:2468] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 1.
I20251210 21:06:37.906817 28891 leader_election.cc:304] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:37.907099 28978 raft_consensus.cc:2804] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:37.907244 28978 raft_consensus.cc:697] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Becoming Leader. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Running, Role: LEADER
I20251210 21:06:37.907516 28978 consensus_queue.cc:237] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:37.910133 28674 catalog_manager.cc:5654] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b reported cstate change: term changed from 0 to 1, leader changed from <none> to ed0eda03e402411db17cfa89ad03980b (127.21.127.67). New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:38.079367 28973 raft_consensus.cc:493] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:38.079557 28973 raft_consensus.cc:515] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.080240 28973 leader_election.cc:290] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.086386 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "15715890f89e4598805c5cc2ff489db3" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" is_pre_election: true
I20251210 21:06:38.086520 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "15715890f89e4598805c5cc2ff489db3" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b" is_pre_election: true
I20251210 21:06:38.086612 28776 raft_consensus.cc:2468] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.086719 28925 raft_consensus.cc:2468] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.086956 28815 leader_election.cc:304] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae; no voters:
I20251210 21:06:38.087195 28973 raft_consensus.cc:2804] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:38.087268 28973 raft_consensus.cc:493] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:38.087344 28973 raft_consensus.cc:3060] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.089262 28973 raft_consensus.cc:515] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.089790 28973 leader_election.cc:290] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Requested vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.090306 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "15715890f89e4598805c5cc2ff489db3" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
I20251210 21:06:38.090345 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "15715890f89e4598805c5cc2ff489db3" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b"
I20251210 21:06:38.090451 28776 raft_consensus.cc:3060] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.090469 28925 raft_consensus.cc:3060] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.092286 28776 raft_consensus.cc:2468] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.092293 28925 raft_consensus.cc:2468] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.092698 28815 leader_election.cc:304] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae; no voters:
I20251210 21:06:38.092939 28973 raft_consensus.cc:2804] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:38.093401 28973 raft_consensus.cc:697] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Becoming Leader. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Running, Role: LEADER
I20251210 21:06:38.093652 28973 consensus_queue.cc:237] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.096201 28674 catalog_manager.cc:5654] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f reported cstate change: term changed from 0 to 1, leader changed from <none> to 00cefb4b862b4514af558a61c220754f (127.21.127.66). New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:38.129630 28973 raft_consensus.cc:493] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:38.129848 28973 raft_consensus.cc:515] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.130506 28973 leader_election.cc:290] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.131108 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18db8ba4df094f5f8e27ae62a446c78a" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" is_pre_election: true
I20251210 21:06:38.131189 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18db8ba4df094f5f8e27ae62a446c78a" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b" is_pre_election: true
I20251210 21:06:38.131304 28776 raft_consensus.cc:2468] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.131393 28925 raft_consensus.cc:2468] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.131693 28815 leader_election.cc:304] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae; no voters:
I20251210 21:06:38.131930 28973 raft_consensus.cc:2804] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:38.132004 28973 raft_consensus.cc:493] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:38.132064 28973 raft_consensus.cc:3060] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.133993 28973 raft_consensus.cc:515] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.134524 28973 leader_election.cc:290] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Requested vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.135061 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18db8ba4df094f5f8e27ae62a446c78a" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b"
I20251210 21:06:38.135057 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18db8ba4df094f5f8e27ae62a446c78a" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
I20251210 21:06:38.135259 28776 raft_consensus.cc:3060] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.135259 28925 raft_consensus.cc:3060] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.138033 28776 raft_consensus.cc:2468] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.138049 28925 raft_consensus.cc:2468] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.138463 28815 leader_election.cc:304] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae; no voters:
I20251210 21:06:38.138689 28973 raft_consensus.cc:2804] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:38.138811 28973 raft_consensus.cc:697] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Becoming Leader. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Running, Role: LEADER
I20251210 21:06:38.139057 28973 consensus_queue.cc:237] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.141806 28674 catalog_manager.cc:5654] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f reported cstate change: term changed from 0 to 1, leader changed from <none> to 00cefb4b862b4514af558a61c220754f (127.21.127.66). New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:38.167331 22013 test_util.cc:276] Using random seed: 1735786774
I20251210 21:06:38.185855 28674 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60504:
name: "dugtrio"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251210 21:06:38.186811 28674 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table dugtrio in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251210 21:06:38.198733 28766 tablet_service.cc:1505] Processing CreateTablet for tablet 3da75733d2f84692be466f98b467fdf8 (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:38.199086 28838 tablet_service.cc:1505] Processing CreateTablet for tablet 3da75733d2f84692be466f98b467fdf8 (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:38.199275 28766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da75733d2f84692be466f98b467fdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.199473 28914 tablet_service.cc:1505] Processing CreateTablet for tablet 3da75733d2f84692be466f98b467fdf8 (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251210 21:06:38.199576 28838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da75733d2f84692be466f98b467fdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.199841 28914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da75733d2f84692be466f98b467fdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.205307 28970 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:38.206043 28766 tablet_service.cc:1505] Processing CreateTablet for tablet bdafef28eb994416a18056a1954371bf (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:38.206660 28766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdafef28eb994416a18056a1954371bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.206600 28841 tablet_service.cc:1505] Processing CreateTablet for tablet bdafef28eb994416a18056a1954371bf (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:38.206782 28913 tablet_service.cc:1505] Processing CreateTablet for tablet bdafef28eb994416a18056a1954371bf (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251210 21:06:38.207074 28841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdafef28eb994416a18056a1954371bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.207101 28913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdafef28eb994416a18056a1954371bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.207262 28970 tablet_bootstrap.cc:654] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.215023 28975 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:38.217125 28975 tablet_bootstrap.cc:654] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.217242 28840 tablet_service.cc:1505] Processing CreateTablet for tablet d97dbbea8e5f41cdb17bf7b603c051aa (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:38.217670 28840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97dbbea8e5f41cdb17bf7b603c051aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.218487 28915 tablet_service.cc:1505] Processing CreateTablet for tablet d97dbbea8e5f41cdb17bf7b603c051aa (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:38.218938 28915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97dbbea8e5f41cdb17bf7b603c051aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.220029 28764 tablet_service.cc:1505] Processing CreateTablet for tablet a9f76563dd904842840a84f4f49707a0 (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:38.220499 28764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f76563dd904842840a84f4f49707a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.226459 28971 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:38.228417 28839 tablet_service.cc:1505] Processing CreateTablet for tablet a9f76563dd904842840a84f4f49707a0 (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:38.228911 28839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f76563dd904842840a84f4f49707a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.229353 28913 tablet_service.cc:1505] Processing CreateTablet for tablet a9f76563dd904842840a84f4f49707a0 (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251210 21:06:38.229791 28970 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:38.229792 28913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f76563dd904842840a84f4f49707a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.229979 28970 ts_tablet_manager.cc:1403] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.015s
I20251210 21:06:38.230821 28970 raft_consensus.cc:359] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.231041 28970 raft_consensus.cc:385] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.231139 28970 raft_consensus.cc:740] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.231417 28970 consensus_queue.cc:260] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.232861 28970 ts_tablet_manager.cc:1434] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251210 21:06:38.233217 28970 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:38.235066 28970 tablet_bootstrap.cc:654] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.237238 28970 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:38.237416 28970 ts_tablet_manager.cc:1403] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251210 21:06:38.238250 28970 raft_consensus.cc:359] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.238512 28970 raft_consensus.cc:385] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.238602 28970 raft_consensus.cc:740] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.238852 28970 consensus_queue.cc:260] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.239423 28970 ts_tablet_manager.cc:1434] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:06:38.239545 28971 tablet_bootstrap.cc:654] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.239814 28970 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:38.240012 28766 tablet_service.cc:1505] Processing CreateTablet for tablet d97dbbea8e5f41cdb17bf7b603c051aa (DEFAULT_TABLE table=dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251210 21:06:38.240522 28766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97dbbea8e5f41cdb17bf7b603c051aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:38.241532 28970 tablet_bootstrap.cc:654] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.246802 28970 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:38.246824 28975 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:38.247079 28970 ts_tablet_manager.cc:1403] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20251210 21:06:38.247077 28975 ts_tablet_manager.cc:1403] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.002s
I20251210 21:06:38.248006 28975 raft_consensus.cc:359] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.248106 28970 raft_consensus.cc:359] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.248220 28975 raft_consensus.cc:385] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.248271 28970 raft_consensus.cc:385] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.248332 28975 raft_consensus.cc:740] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.248361 28970 raft_consensus.cc:740] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.248570 28970 consensus_queue.cc:260] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.249107 28970 ts_tablet_manager.cc:1434] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:38.249317 28975 consensus_queue.cc:260] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.249883 28970 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:38.251837 28970 tablet_bootstrap.cc:654] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.252394 28975 ts_tablet_manager.cc:1434] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251210 21:06:38.252779 28975 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:38.254766 28975 tablet_bootstrap.cc:654] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.260375 28971 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:38.260524 28975 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:38.260586 28971 ts_tablet_manager.cc:1403] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.000s
I20251210 21:06:38.260807 28975 ts_tablet_manager.cc:1403] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20251210 21:06:38.261380 28970 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: No bootstrap required, opened a new log
I20251210 21:06:38.261418 28971 raft_consensus.cc:359] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.261564 28970 ts_tablet_manager.cc:1403] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20251210 21:06:38.261633 28971 raft_consensus.cc:385] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.261631 28975 raft_consensus.cc:359] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.261799 28971 raft_consensus.cc:740] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.261857 28975 raft_consensus.cc:385] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.261942 28975 raft_consensus.cc:740] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.262108 28971 consensus_queue.cc:260] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.262188 28975 consensus_queue.cc:260] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.262333 28970 raft_consensus.cc:359] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.262516 28970 raft_consensus.cc:385] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.262624 28970 raft_consensus.cc:740] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.262800 28975 ts_tablet_manager.cc:1434] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251210 21:06:38.262904 28970 consensus_queue.cc:260] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.263460 28975 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:38.263751 28970 ts_tablet_manager.cc:1434] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:06:38.265286 28971 ts_tablet_manager.cc:1434] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251210 21:06:38.265631 28975 tablet_bootstrap.cc:654] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.265682 28971 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:38.270252 28971 tablet_bootstrap.cc:654] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.275034 28975 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:38.275226 28975 ts_tablet_manager.cc:1403] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20251210 21:06:38.275295 28971 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:38.275553 28971 ts_tablet_manager.cc:1403] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20251210 21:06:38.276340 28971 raft_consensus.cc:359] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.276440 28975 raft_consensus.cc:359] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.276593 28971 raft_consensus.cc:385] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.276646 28975 raft_consensus.cc:385] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.276690 28971 raft_consensus.cc:740] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.276758 28975 raft_consensus.cc:740] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.276952 28971 consensus_queue.cc:260] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.276984 28975 consensus_queue.cc:260] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.277487 28975 ts_tablet_manager.cc:1434] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251210 21:06:38.277485 28971 ts_tablet_manager.cc:1434] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:06:38.277798 28975 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: Bootstrap starting.
I20251210 21:06:38.277859 28971 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:38.279731 28975 tablet_bootstrap.cc:654] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.280846 28971 tablet_bootstrap.cc:654] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.281706 28975 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: No bootstrap required, opened a new log
I20251210 21:06:38.281965 28975 ts_tablet_manager.cc:1403] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20251210 21:06:38.282641 28971 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:38.282789 28971 ts_tablet_manager.cc:1403] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:06:38.282897 28975 raft_consensus.cc:359] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.283110 28975 raft_consensus.cc:385] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.283201 28975 raft_consensus.cc:740] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.283473 28975 consensus_queue.cc:260] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.283567 28971 raft_consensus.cc:359] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.283753 28971 raft_consensus.cc:385] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.283852 28971 raft_consensus.cc:740] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.283978 28975 ts_tablet_manager.cc:1434] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251210 21:06:38.284075 28971 consensus_queue.cc:260] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.284580 28971 ts_tablet_manager.cc:1434] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:38.284898 28971 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: Bootstrap starting.
I20251210 21:06:38.286862 28971 tablet_bootstrap.cc:654] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: Neither blocks nor log segments found. Creating new log.
I20251210 21:06:38.292505 28971 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: No bootstrap required, opened a new log
I20251210 21:06:38.292699 28971 ts_tablet_manager.cc:1403] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20251210 21:06:38.293385 28971 raft_consensus.cc:359] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.293573 28971 raft_consensus.cc:385] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251210 21:06:38.293651 28971 raft_consensus.cc:740] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Initialized, Role: FOLLOWER
I20251210 21:06:38.293875 28971 consensus_queue.cc:260] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.294309 28971 ts_tablet_manager.cc:1434] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251210 21:06:38.317658 28973 raft_consensus.cc:493] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:38.317869 28973 raft_consensus.cc:515] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.318545 28973 leader_election.cc:290] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.319128 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bdafef28eb994416a18056a1954371bf" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" is_pre_election: true
I20251210 21:06:38.319175 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bdafef28eb994416a18056a1954371bf" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b" is_pre_election: true
I20251210 21:06:38.319396 28776 raft_consensus.cc:2468] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.319523 28925 raft_consensus.cc:2468] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.319847 28815 leader_election.cc:304] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae; no voters:
I20251210 21:06:38.320130 28973 raft_consensus.cc:2804] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:38.320235 28973 raft_consensus.cc:493] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:38.320338 28973 raft_consensus.cc:3060] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.322495 28973 raft_consensus.cc:515] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.323555 28973 leader_election.cc:290] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Requested vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.323935 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bdafef28eb994416a18056a1954371bf" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
I20251210 21:06:38.324077 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bdafef28eb994416a18056a1954371bf" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b"
I20251210 21:06:38.324163 28776 raft_consensus.cc:3060] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.324345 28925 raft_consensus.cc:3060] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.325369 28978 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:38.327178 28990 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:38.327777 28925 raft_consensus.cc:2468] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.327962 28776 raft_consensus.cc:2468] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.328250 28815 leader_election.cc:304] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:38.328847 28973 raft_consensus.cc:2804] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:38.329115 28973 raft_consensus.cc:697] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Becoming Leader. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Running, Role: LEADER
I20251210 21:06:38.329475 28973 consensus_queue.cc:237] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.347023 28978 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:38.353399 28978 raft_consensus.cc:493] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:38.353680 28978 raft_consensus.cc:515] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.354332 28674 catalog_manager.cc:5654] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f reported cstate change: term changed from 0 to 1, leader changed from <none> to 00cefb4b862b4514af558a61c220754f (127.21.127.66). New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:38.354815 28850 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9f76563dd904842840a84f4f49707a0" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f" is_pre_election: true
I20251210 21:06:38.355023 28850 raft_consensus.cc:2468] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 0.
I20251210 21:06:38.355428 28891 leader_election.cc:304] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:38.355543 28775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9f76563dd904842840a84f4f49707a0" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" is_pre_election: true
I20251210 21:06:38.355757 28775 raft_consensus.cc:2468] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 0.
I20251210 21:06:38.355857 29023 raft_consensus.cc:2804] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:38.355959 29023 raft_consensus.cc:493] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:38.356079 29023 raft_consensus.cc:3060] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.355763 29018 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20251210 21:06:38.358971 29023 raft_consensus.cc:515] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.360119 28850 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9f76563dd904842840a84f4f49707a0" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f"
I20251210 21:06:38.360308 28850 raft_consensus.cc:3060] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.361498 28775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9f76563dd904842840a84f4f49707a0" candidate_uuid: "ed0eda03e402411db17cfa89ad03980b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
I20251210 21:06:38.361696 28775 raft_consensus.cc:3060] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.363308 28850 raft_consensus.cc:2468] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 1.
I20251210 21:06:38.363798 28891 leader_election.cc:304] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:38.364128 28978 leader_election.cc:290] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:38.364406 28978 raft_consensus.cc:2804] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:38.364557 28978 raft_consensus.cc:697] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Becoming Leader. State: Replica: ed0eda03e402411db17cfa89ad03980b, State: Running, Role: LEADER
I20251210 21:06:38.364696 28775 raft_consensus.cc:2468] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed0eda03e402411db17cfa89ad03980b in term 1.
I20251210 21:06:38.364904 28978 consensus_queue.cc:237] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.366036 29023 leader_election.cc:290] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [CANDIDATE]: Term 1 election: Requested vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:38.368037 28674 catalog_manager.cc:5654] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b reported cstate change: term changed from 0 to 1, leader changed from <none> to ed0eda03e402411db17cfa89ad03980b (127.21.127.67). New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:38.454329 28973 raft_consensus.cc:493] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:38.454612 28973 raft_consensus.cc:515] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.455427 28973 leader_election.cc:290] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.456081 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3da75733d2f84692be466f98b467fdf8" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" is_pre_election: true
I20251210 21:06:38.456091 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3da75733d2f84692be466f98b467fdf8" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b" is_pre_election: true
I20251210 21:06:38.456339 28776 raft_consensus.cc:2468] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.456369 28925 raft_consensus.cc:2468] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 0.
I20251210 21:06:38.456777 28815 leader_election.cc:304] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:38.457096 28973 raft_consensus.cc:2804] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:38.457185 28973 raft_consensus.cc:493] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:38.457290 28973 raft_consensus.cc:3060] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.459254 28973 raft_consensus.cc:515] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.459901 28973 leader_election.cc:290] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Requested vote from peers 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.460448 28776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3da75733d2f84692be466f98b467fdf8" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
I20251210 21:06:38.460498 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3da75733d2f84692be466f98b467fdf8" candidate_uuid: "00cefb4b862b4514af558a61c220754f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b"
I20251210 21:06:38.460656 28776 raft_consensus.cc:3060] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.460680 28925 raft_consensus.cc:3060] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.463002 28925 raft_consensus.cc:2468] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.463109 28776 raft_consensus.cc:2468] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cefb4b862b4514af558a61c220754f in term 1.
I20251210 21:06:38.463430 28815 leader_election.cc:304] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:38.463742 28973 raft_consensus.cc:2804] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:38.463891 28973 raft_consensus.cc:697] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Becoming Leader. State: Replica: 00cefb4b862b4514af558a61c220754f, State: Running, Role: LEADER
I20251210 21:06:38.464175 28973 consensus_queue.cc:237] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.466907 28674 catalog_manager.cc:5654] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f reported cstate change: term changed from 0 to 1, leader changed from <none> to 00cefb4b862b4514af558a61c220754f (127.21.127.66). New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: UNKNOWN } } }
W20251210 21:06:38.467339 28722 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20251210 21:06:38.519695 28973 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:38.524259 29031 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:38.568295 29031 consensus_queue.cc:1048] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:38.572816 28973 consensus_queue.cc:1048] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251210 21:06:38.603225 29026 raft_consensus.cc:493] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251210 21:06:38.603471 29026 raft_consensus.cc:515] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.604315 29026 leader_election.cc:290] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.620811 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d97dbbea8e5f41cdb17bf7b603c051aa" candidate_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b" is_pre_election: true
I20251210 21:06:38.621102 28850 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d97dbbea8e5f41cdb17bf7b603c051aa" candidate_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f" is_pre_election: true
I20251210 21:06:38.621111 28925 raft_consensus.cc:2468] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e0cab8636c34a3eabffe2495f82e1ae in term 0.
I20251210 21:06:38.621300 28850 raft_consensus.cc:2468] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e0cab8636c34a3eabffe2495f82e1ae in term 0.
I20251210 21:06:38.621551 28740 leader_election.cc:304] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e0cab8636c34a3eabffe2495f82e1ae, ed0eda03e402411db17cfa89ad03980b; no voters:
I20251210 21:06:38.621799 29026 raft_consensus.cc:2804] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251210 21:06:38.621883 29026 raft_consensus.cc:493] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251210 21:06:38.621987 29026 raft_consensus.cc:3060] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.623934 29026 raft_consensus.cc:515] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.624491 29026 leader_election.cc:290] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [CANDIDATE]: Term 1 election: Requested vote from peers 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303), ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:38.624979 28925 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d97dbbea8e5f41cdb17bf7b603c051aa" candidate_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0eda03e402411db17cfa89ad03980b"
I20251210 21:06:38.625038 28850 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d97dbbea8e5f41cdb17bf7b603c051aa" candidate_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cefb4b862b4514af558a61c220754f"
I20251210 21:06:38.625133 28925 raft_consensus.cc:3060] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.625216 28850 raft_consensus.cc:3060] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 0 FOLLOWER]: Advancing to term 1
I20251210 21:06:38.627147 28850 raft_consensus.cc:2468] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e0cab8636c34a3eabffe2495f82e1ae in term 1.
I20251210 21:06:38.627133 28925 raft_consensus.cc:2468] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e0cab8636c34a3eabffe2495f82e1ae in term 1.
I20251210 21:06:38.627605 28742 leader_election.cc:304] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cefb4b862b4514af558a61c220754f, 0e0cab8636c34a3eabffe2495f82e1ae; no voters:
I20251210 21:06:38.627879 29026 raft_consensus.cc:2804] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Leader election won for term 1
I20251210 21:06:38.628007 29026 raft_consensus.cc:697] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 LEADER]: Becoming Leader. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Running, Role: LEADER
I20251210 21:06:38.628245 29026 consensus_queue.cc:237] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:38.630925 28673 catalog_manager.cc:5654] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65). New cstate: current_term: 1 leader_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:38.680143 22013 tablet_server.cc:178] TabletServer@127.21.127.65:0 shutting down...
I20251210 21:06:38.688601 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:38.689102 22013 tablet_replica.cc:333] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.689357 22013 raft_consensus.cc:2243] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:38.689649 22013 pending_rounds.cc:70] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Trying to abort 1 pending ops.
I20251210 21:06:38.689699 22013 pending_rounds.cc:77] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081671183527936 op_type: NO_OP noop_request { }
I20251210 21:06:38.689810 22013 raft_consensus.cc:2889] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:38.689914 22013 raft_consensus.cc:2272] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.690838 22013 tablet_replica.cc:333] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.691008 22013 raft_consensus.cc:2243] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:38.691141 22013 raft_consensus.cc:2272] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.691751 22013 tablet_replica.cc:333] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.691876 22013 raft_consensus.cc:2243] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:38.692015 22013 raft_consensus.cc:2272] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.692569 22013 tablet_replica.cc:333] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.692692 22013 raft_consensus.cc:2243] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:38.692821 22013 raft_consensus.cc:2272] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.693428 22013 tablet_replica.cc:333] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.693555 22013 raft_consensus.cc:2243] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:38.693670 22013 raft_consensus.cc:2272] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.694205 22013 tablet_replica.cc:333] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.694339 22013 raft_consensus.cc:2243] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:38.694464 22013 raft_consensus.cc:2272] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.695097 22013 tablet_replica.cc:333] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.695199 22013 raft_consensus.cc:2243] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:38.695341 22013 raft_consensus.cc:2272] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.695931 22013 tablet_replica.cc:333] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: stopping tablet replica
I20251210 21:06:38.696039 22013 raft_consensus.cc:2243] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:38.696146 22013 raft_consensus.cc:2272] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:38.712250 22013 tablet_server.cc:195] TabletServer@127.21.127.65:0 shutdown complete.
I20251210 21:06:38.775915 28978 consensus_queue.cc:1048] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:06:38.782620 28889 consensus_peers.cc:597] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251210 21:06:38.838014 28889 consensus_peers.cc:597] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251210 21:06:38.865217 28889 consensus_peers.cc:597] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:38.887559 28973 consensus_queue.cc:1048] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:06:38.893801 28815 consensus_peers.cc:597] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:38.914652 29044 consensus_queue.cc:1048] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251210 21:06:38.920178 28815 consensus_peers.cc:597] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251210 21:06:39.075188 28815 consensus_peers.cc:597] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251210 21:06:39.142894 28815 consensus_peers.cc:597] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.579878 29044 consensus_queue.cc:579] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Leader has been unable to successfully communicate with peer 0e0cab8636c34a3eabffe2495f82e1ae for more than 1 seconds (1.052s)
I20251210 21:06:39.685746 29018 consensus_queue.cc:579] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Leader has been unable to successfully communicate with peer 0e0cab8636c34a3eabffe2495f82e1ae for more than 1 seconds (1.327s)
I20251210 21:06:39.706841 28973 consensus_queue.cc:579] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Leader has been unable to successfully communicate with peer 0e0cab8636c34a3eabffe2495f82e1ae for more than 1 seconds (1.131s)
I20251210 21:06:39.725824 28674 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60500:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251210 21:06:39.726070 28674 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60500:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251210 21:06:39.731318 28674 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: Sending DeleteTablet for 3 replicas of tablet 3da75733d2f84692be466f98b467fdf8
I20251210 21:06:39.731909 28674 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: Sending DeleteTablet for 3 replicas of tablet bdafef28eb994416a18056a1954371bf
I20251210 21:06:39.732318 28674 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: Sending DeleteTablet for 3 replicas of tablet d97dbbea8e5f41cdb17bf7b603c051aa
I20251210 21:06:39.732462 28840 tablet_service.cc:1552] Processing DeleteTablet for tablet 3da75733d2f84692be466f98b467fdf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:41142
I20251210 21:06:39.732714 28674 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: Sending DeleteTablet for 3 replicas of tablet a9f76563dd904842840a84f4f49707a0
I20251210 21:06:39.733369 28841 tablet_service.cc:1552] Processing DeleteTablet for tablet bdafef28eb994416a18056a1954371bf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:41142
I20251210 21:06:39.733404 28915 tablet_service.cc:1552] Processing DeleteTablet for tablet 3da75733d2f84692be466f98b467fdf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:39718
I20251210 21:06:39.733670 28838 tablet_service.cc:1552] Processing DeleteTablet for tablet d97dbbea8e5f41cdb17bf7b603c051aa with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:41142
I20251210 21:06:39.733722 28913 tablet_service.cc:1552] Processing DeleteTablet for tablet bdafef28eb994416a18056a1954371bf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:39718
I20251210 21:06:39.733896 28914 tablet_service.cc:1552] Processing DeleteTablet for tablet d97dbbea8e5f41cdb17bf7b603c051aa with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:39718
I20251210 21:06:39.734440 28838 tablet_service.cc:1552] Processing DeleteTablet for tablet a9f76563dd904842840a84f4f49707a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:41142
I20251210 21:06:39.734890 29055 tablet_replica.cc:333] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:39.734891 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251210 21:06:39.735183 29055 raft_consensus.cc:2243] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:39.735196 29056 tablet_replica.cc:333] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:39.735509 29056 raft_consensus.cc:2243] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:39.735208 28914 tablet_service.cc:1552] Processing DeleteTablet for tablet a9f76563dd904842840a84f4f49707a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:39718
I20251210 21:06:39.735633 29055 raft_consensus.cc:2272] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.735687 29056 raft_consensus.cc:2272] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.736431 29055 ts_tablet_manager.cc:1916] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.736778 29056 ts_tablet_manager.cc:1916] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251210 21:06:39.742023 28659 catalog_manager.cc:4712] TS 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a9f76563dd904842840a84f4f49707a0: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111)
W20251210 21:06:39.742506 29058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:39.743665 29055 ts_tablet_manager.cc:1929] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251210 21:06:39.743831 29055 log.cc:1199] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/3da75733d2f84692be466f98b467fdf8
I20251210 21:06:39.744206 29056 ts_tablet_manager.cc:1929] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251210 21:06:39.744289 29055 ts_tablet_manager.cc:1950] T 3da75733d2f84692be466f98b467fdf8 P 00cefb4b862b4514af558a61c220754f: Deleting consensus metadata
I20251210 21:06:39.744357 29056 log.cc:1199] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/3da75733d2f84692be466f98b467fdf8
I20251210 21:06:39.744771 29056 ts_tablet_manager.cc:1950] T 3da75733d2f84692be466f98b467fdf8 P ed0eda03e402411db17cfa89ad03980b: Deleting consensus metadata
I20251210 21:06:39.745781 28661 catalog_manager.cc:4985] TS 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303): tablet 3da75733d2f84692be466f98b467fdf8 (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.745857 28659 catalog_manager.cc:4985] TS ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227): tablet 3da75733d2f84692be466f98b467fdf8 (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
W20251210 21:06:39.745901 29059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:39.746136 29055 tablet_replica.cc:333] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:39.746253 29056 tablet_replica.cc:333] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:39.746417 29055 raft_consensus.cc:2243] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:39.746505 29056 raft_consensus.cc:2243] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:39.746728 29056 raft_consensus.cc:2272] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.746829 29055 raft_consensus.cc:2272] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.747879 29055 ts_tablet_manager.cc:1916] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.747854 29056 ts_tablet_manager.cc:1916] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.755698 29055 ts_tablet_manager.cc:1929] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251210 21:06:39.755882 29055 log.cc:1199] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/bdafef28eb994416a18056a1954371bf
I20251210 21:06:39.756095 29056 ts_tablet_manager.cc:1929] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251210 21:06:39.756208 29056 log.cc:1199] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/bdafef28eb994416a18056a1954371bf
I20251210 21:06:39.756228 29055 ts_tablet_manager.cc:1950] T bdafef28eb994416a18056a1954371bf P 00cefb4b862b4514af558a61c220754f: Deleting consensus metadata
I20251210 21:06:39.756592 29056 ts_tablet_manager.cc:1950] T bdafef28eb994416a18056a1954371bf P ed0eda03e402411db17cfa89ad03980b: Deleting consensus metadata
I20251210 21:06:39.756942 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:39.757403 28661 catalog_manager.cc:4985] TS 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303): tablet bdafef28eb994416a18056a1954371bf (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.757548 29056 tablet_replica.cc:333] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:39.757673 28659 catalog_manager.cc:4985] TS ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227): tablet bdafef28eb994416a18056a1954371bf (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.757707 29055 tablet_replica.cc:333] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:39.757730 29056 raft_consensus.cc:2243] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:39.757901 29055 raft_consensus.cc:2243] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:39.758054 29056 raft_consensus.cc:2272] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.758051 29055 raft_consensus.cc:2272] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
W20251210 21:06:39.758363 29061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:39.758667 29055 ts_tablet_manager.cc:1916] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.758673 29056 ts_tablet_manager.cc:1916] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.758786 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:39.758862 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:39.758901 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400799758897 us; error 0 us; skew 500 ppm
I20251210 21:06:39.759052 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:39.760556 22013 webserver.cc:492] Webserver started at http://127.21.127.68:38351/ using document root <none> and password file <none>
I20251210 21:06:39.760826 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:39.760900 22013 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251210 21:06:39.761022 22013 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251210 21:06:39.761696 22013 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root/instance:
uuid: "6f9f3e8755674168af3060a746b58c24"
format_stamp: "Formatted at 2025-12-10 21:06:39 on dist-test-slave-9pq8"
I20251210 21:06:39.763707 29055 ts_tablet_manager.cc:1929] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251210 21:06:39.763803 29055 log.cc:1199] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/d97dbbea8e5f41cdb17bf7b603c051aa
I20251210 21:06:39.764062 29055 ts_tablet_manager.cc:1950] T d97dbbea8e5f41cdb17bf7b603c051aa P 00cefb4b862b4514af558a61c220754f: Deleting consensus metadata
I20251210 21:06:39.764878 29056 ts_tablet_manager.cc:1929] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251210 21:06:39.764988 29056 log.cc:1199] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/d97dbbea8e5f41cdb17bf7b603c051aa
I20251210 21:06:39.765000 29055 tablet_replica.cc:333] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:39.765170 29055 raft_consensus.cc:2243] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:39.765264 29056 ts_tablet_manager.cc:1950] T d97dbbea8e5f41cdb17bf7b603c051aa P ed0eda03e402411db17cfa89ad03980b: Deleting consensus metadata
I20251210 21:06:39.765226 28661 catalog_manager.cc:4985] TS 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303): tablet d97dbbea8e5f41cdb17bf7b603c051aa (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.765336 29055 raft_consensus.cc:2272] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.765568 22013 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251210 21:06:39.766084 29055 ts_tablet_manager.cc:1916] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.766162 29056 tablet_replica.cc:333] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:39.766325 29056 raft_consensus.cc:2243] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:39.766343 28659 catalog_manager.cc:4985] TS ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227): tablet d97dbbea8e5f41cdb17bf7b603c051aa (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.766660 29056 raft_consensus.cc:2272] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.767416 29056 ts_tablet_manager.cc:1916] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.768231 29066 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:39.768604 22013 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:39.768704 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root
uuid: "6f9f3e8755674168af3060a746b58c24"
format_stamp: "Formatted at 2025-12-10 21:06:39 on dist-test-slave-9pq8"
I20251210 21:06:39.768829 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:39.771402 29055 ts_tablet_manager.cc:1929] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251210 21:06:39.771567 29055 log.cc:1199] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/a9f76563dd904842840a84f4f49707a0
I20251210 21:06:39.771914 29055 ts_tablet_manager.cc:1950] T a9f76563dd904842840a84f4f49707a0 P 00cefb4b862b4514af558a61c220754f: Deleting consensus metadata
I20251210 21:06:39.772923 28661 catalog_manager.cc:4985] TS 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303): tablet a9f76563dd904842840a84f4f49707a0 (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.773810 29056 ts_tablet_manager.cc:1929] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251210 21:06:39.773926 29056 log.cc:1199] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/a9f76563dd904842840a84f4f49707a0
I20251210 21:06:39.774326 29056 ts_tablet_manager.cc:1950] T a9f76563dd904842840a84f4f49707a0 P ed0eda03e402411db17cfa89ad03980b: Deleting consensus metadata
I20251210 21:06:39.775404 28659 catalog_manager.cc:4985] TS ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227): tablet a9f76563dd904842840a84f4f49707a0 (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.790903 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:39.791569 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:39.792207 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:39.793109 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251210 21:06:39.793164 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:39.793215 22013 ts_tablet_manager.cc:616] Registered 0 tablets
I20251210 21:06:39.793246 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:39.806278 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.68:39083
I20251210 21:06:39.806313 29128 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.68:39083 every 8 connection(s)
I20251210 21:06:39.807755 22013 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251210 21:06:39.810408 29132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:39.810695 29133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251210 21:06:39.816534 29135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251210 21:06:39.817045 22013 server_base.cc:1047] running on GCE node
I20251210 21:06:39.817271 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251210 21:06:39.817333 22013 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251210 21:06:39.817358 29129 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44701
I20251210 21:06:39.817364 22013 hybrid_clock.cc:648] HybridClock initialized: now 1765400799817360 us; error 0 us; skew 500 ppm
I20251210 21:06:39.817493 29129 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:39.817560 22013 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251210 21:06:39.817874 29129 heartbeater.cc:507] Master 127.21.127.126:44701 requested a full tablet report, sending...
I20251210 21:06:39.818920 28674 ts_manager.cc:194] Registered new tserver with Master: 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083)
I20251210 21:06:39.818953 22013 webserver.cc:492] Webserver started at http://127.21.127.65:41797/ using document root <none> and password file <none>
I20251210 21:06:39.819216 22013 fs_manager.cc:362] Metadata directory not provided
I20251210 21:06:39.819299 22013 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251210 21:06:39.820063 28674 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60518
I20251210 21:06:39.821705 22013 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:39.823383 29140 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251210 21:06:39.823812 22013 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251210 21:06:39.823913 22013 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root
uuid: "0e0cab8636c34a3eabffe2495f82e1ae"
format_stamp: "Formatted at 2025-12-10 21:06:36 on dist-test-slave-9pq8"
I20251210 21:06:39.824043 22013 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251210 21:06:39.824191 29018 consensus_queue.cc:579] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Leader has been unable to successfully communicate with peer 0e0cab8636c34a3eabffe2495f82e1ae for more than 1 seconds (1.457s)
I20251210 21:06:39.828233 28925 consensus_queue.cc:237] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:39.830909 28850 consensus_queue.cc:237] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:39.831184 28851 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:06:39.831808 28978 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:06:39.834805 22013 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251210 21:06:39.835539 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251210 21:06:39.835556 28925 raft_consensus.cc:1275] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:06:39.850189 28925 consensus_queue.cc:237] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:39.854707 28978 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:39.859030 28659 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 8dcdcf8eac50460b83c74ee9a90520e7 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251210 21:06:39.859462 22013 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251210 21:06:39.860122 28850 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
W20251210 21:06:39.861302 28889 consensus_peers.cc:597] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.866904 28674 catalog_manager.cc:5654] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:39.868911 28850 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251210 21:06:39.869526 29018 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:06:39.873471 28995 consensus_queue.cc:1048] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251210 21:06:39.874265 28815 consensus_peers.cc:597] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.874567 29018 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
W20251210 21:06:39.875591 28889 consensus_peers.cc:597] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.878978 28659 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 94d66b996d8742b192d3ab2b22decb5d with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 4)
I20251210 21:06:39.880654 28850 consensus_queue.cc:237] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:39.880810 28849 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:39.882789 28973 raft_consensus.cc:2955] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
W20251210 21:06:39.884549 28816 consensus_peers.cc:597] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f -> Peer 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): Couldn't send request to peer 6f9f3e8755674168af3060a746b58c24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18db8ba4df094f5f8e27ae62a446c78a. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.885774 28674 catalog_manager.cc:5654] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:39.887166 28661 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 18db8ba4df094f5f8e27ae62a446c78a with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 3)
W20251210 21:06:39.889642 28816 consensus_peers.cc:597] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f -> Peer 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): Couldn't send request to peer 6f9f3e8755674168af3060a746b58c24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15715890f89e4598805c5cc2ff489db3. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.889343 28924 raft_consensus.cc:2955] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:39.890995 28923 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251210 21:06:39.891517 28815 consensus_peers.cc:597] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.891914 28973 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251210 21:06:39.895475 28673 catalog_manager.cc:5654] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:39.897457 29160 ts_tablet_manager.cc:548] Loading tablet metadata (0/8 complete)
W20251210 21:06:39.898790 28890 consensus_peers.cc:597] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b -> Peer 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): Couldn't send request to peer 6f9f3e8755674168af3060a746b58c24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8dcdcf8eac50460b83c74ee9a90520e7. This is attempt 1: this message will repeat every 5th retry.
W20251210 21:06:39.899153 28890 consensus_peers.cc:597] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b -> Peer 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): Couldn't send request to peer 6f9f3e8755674168af3060a746b58c24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94d66b996d8742b192d3ab2b22decb5d. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:39.900520 29044 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:39.901881 28924 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:39.905064 28661 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 15715890f89e4598805c5cc2ff489db3 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 5)
I20251210 21:06:39.906173 28674 catalog_manager.cc:5654] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:39.929962 22013 ts_tablet_manager.cc:585] Loaded tablet metadata (8 total tablets, 8 live tablets)
I20251210 21:06:39.930100 22013 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.037s user 0.005s sys 0.000s
I20251210 21:06:39.930198 22013 ts_tablet_manager.cc:600] Registering tablets (0/8 complete)
I20251210 21:06:39.932324 29160 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.936407 29160 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:39.936756 29160 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.936975 29160 ts_tablet_manager.cc:1403] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:06:39.937537 29160 raft_consensus.cc:359] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.937690 29160 raft_consensus.cc:740] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.937902 29160 consensus_queue.cc:260] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.938503 29160 ts_tablet_manager.cc:1434] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.003s
I20251210 21:06:39.938709 29160 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.942258 29160 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:39.942590 29160 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.942792 29160 ts_tablet_manager.cc:1403] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251210 21:06:39.943341 29160 raft_consensus.cc:359] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.943521 29160 raft_consensus.cc:740] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.943719 29160 consensus_queue.cc:260] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.944094 29160 ts_tablet_manager.cc:1434] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:39.944175 22013 ts_tablet_manager.cc:616] Registered 8 tablets
I20251210 21:06:39.944248 22013 ts_tablet_manager.cc:595] Time spent register tablets: real 0.014s user 0.014s sys 0.000s
I20251210 21:06:39.944314 29160 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.947495 29160 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:39.947853 29160 tablet_bootstrap.cc:492] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.948098 29160 ts_tablet_manager.cc:1403] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:39.948763 29160 raft_consensus.cc:359] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.948917 29160 raft_consensus.cc:740] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.949188 29160 consensus_queue.cc:260] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.949633 29160 ts_tablet_manager.cc:1434] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:39.949848 29160 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.953709 29160 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:39.954103 29160 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.954335 29160 ts_tablet_manager.cc:1403] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251210 21:06:39.954926 29160 raft_consensus.cc:359] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.955072 29160 raft_consensus.cc:740] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.955271 29160 consensus_queue.cc:260] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.955693 29160 ts_tablet_manager.cc:1434] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251210 21:06:39.955904 29160 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.959415 29160 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251210 21:06:39.959827 29160 tablet_bootstrap.cc:492] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.960101 29160 ts_tablet_manager.cc:1403] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251210 21:06:39.960811 29160 raft_consensus.cc:359] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.961025 29160 raft_consensus.cc:740] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.961256 29160 consensus_queue.cc:260] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.961630 29160 ts_tablet_manager.cc:1434] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:39.961849 29160 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.966034 29160 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:39.966440 29160 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.966677 29160 ts_tablet_manager.cc:1403] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251210 21:06:39.967383 29160 raft_consensus.cc:359] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.967643 29160 raft_consensus.cc:740] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.967859 29160 consensus_queue.cc:260] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.968240 29160 ts_tablet_manager.cc:1434] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251210 21:06:39.968454 29160 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.971746 29160 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:39.972160 29160 tablet_bootstrap.cc:492] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.972405 29160 ts_tablet_manager.cc:1403] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251210 21:06:39.973037 29160 raft_consensus.cc:359] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.973217 29160 raft_consensus.cc:740] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.973431 29160 consensus_queue.cc:260] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.973882 29160 ts_tablet_manager.cc:1434] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:39.973960 22013 rpc_server.cc:307] RPC server started. Bound to: 127.21.127.65:43855
I20251210 21:06:39.973985 29224 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.127.65:43855 every 8 connection(s)
I20251210 21:06:39.974140 29160 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap starting.
I20251210 21:06:39.978457 29160 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:39.978812 29160 tablet_bootstrap.cc:492] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Bootstrap complete.
I20251210 21:06:39.979100 29160 ts_tablet_manager.cc:1403] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251210 21:06:39.979895 29160 raft_consensus.cc:359] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.980052 29160 raft_consensus.cc:740] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cab8636c34a3eabffe2495f82e1ae, State: Initialized, Role: FOLLOWER
I20251210 21:06:39.980283 29160 consensus_queue.cc:260] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:39.980888 29160 ts_tablet_manager.cc:1434] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251210 21:06:39.983006 29225 heartbeater.cc:344] Connected to a master server at 127.21.127.126:44701
I20251210 21:06:39.983142 29225 heartbeater.cc:461] Registering TS with master...
I20251210 21:06:39.983107 29188 tablet_service.cc:1552] Processing DeleteTablet for tablet a9f76563dd904842840a84f4f49707a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:40878
I20251210 21:06:39.983624 29225 heartbeater.cc:507] Master 127.21.127.126:44701 requested a full tablet report, sending...
I20251210 21:06:39.983780 29230 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:39.983942 29230 raft_consensus.cc:2243] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:39.984102 29230 raft_consensus.cc:2272] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:39.984761 29230 ts_tablet_manager.cc:1916] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:39.985244 28673 ts_manager.cc:194] Re-registered known tserver with Master: 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855)
I20251210 21:06:39.987741 28673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60526
I20251210 21:06:39.988924 29225 heartbeater.cc:499] Master 127.21.127.126:44701 was elected leader, sending a full tablet report...
I20251210 21:06:39.989307 29230 ts_tablet_manager.cc:1929] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251210 21:06:39.989394 29230 log.cc:1199] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/a9f76563dd904842840a84f4f49707a0
I20251210 21:06:39.989631 29230 ts_tablet_manager.cc:1950] T a9f76563dd904842840a84f4f49707a0 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting consensus metadata
I20251210 21:06:39.990706 28659 catalog_manager.cc:4985] TS 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): tablet a9f76563dd904842840a84f4f49707a0 (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:39.992736 29188 tablet_service.cc:1552] Processing DeleteTablet for tablet a9f76563dd904842840a84f4f49707a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:40878
W20251210 21:06:39.993248 28659 catalog_manager.cc:4952] TS 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): delete failed for tablet a9f76563dd904842840a84f4f49707a0 because the tablet was not found. No further retry: Not found: Tablet not found: a9f76563dd904842840a84f4f49707a0
I20251210 21:06:40.078521 29188 tablet_service.cc:1552] Processing DeleteTablet for tablet 3da75733d2f84692be466f98b467fdf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:40878
I20251210 21:06:40.078856 29230 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:40.079068 29230 raft_consensus.cc:2243] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:40.079233 29230 raft_consensus.cc:2272] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:40.079964 29230 ts_tablet_manager.cc:1916] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:40.084420 29230 ts_tablet_manager.cc:1929] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251210 21:06:40.084532 29230 log.cc:1199] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/3da75733d2f84692be466f98b467fdf8
I20251210 21:06:40.084780 29230 ts_tablet_manager.cc:1950] T 3da75733d2f84692be466f98b467fdf8 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting consensus metadata
I20251210 21:06:40.085718 28659 catalog_manager.cc:4985] TS 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): tablet 3da75733d2f84692be466f98b467fdf8 (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:40.087694 29188 tablet_service.cc:1552] Processing DeleteTablet for tablet d97dbbea8e5f41cdb17bf7b603c051aa with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:40878
I20251210 21:06:40.087999 29230 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:40.088173 29230 raft_consensus.cc:2243] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:40.088328 29230 pending_rounds.cc:70] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Trying to abort 1 pending ops.
I20251210 21:06:40.088367 29230 pending_rounds.cc:77] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231081671183527936 op_type: NO_OP noop_request { }
I20251210 21:06:40.088485 29230 raft_consensus.cc:2889] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251210 21:06:40.088666 29230 raft_consensus.cc:2272] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:40.089483 29230 ts_tablet_manager.cc:1916] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:40.093886 29230 ts_tablet_manager.cc:1929] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251210 21:06:40.094018 29230 log.cc:1199] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/d97dbbea8e5f41cdb17bf7b603c051aa
I20251210 21:06:40.094296 29230 ts_tablet_manager.cc:1950] T d97dbbea8e5f41cdb17bf7b603c051aa P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting consensus metadata
I20251210 21:06:40.095304 28659 catalog_manager.cc:4985] TS 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): tablet d97dbbea8e5f41cdb17bf7b603c051aa (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:40.108587 29188 tablet_service.cc:1552] Processing DeleteTablet for tablet bdafef28eb994416a18056a1954371bf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-10 21:06:39 UTC) from {username='slave'} at 127.0.0.1:40878
I20251210 21:06:40.108923 29230 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:40.109133 29230 raft_consensus.cc:2243] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:40.109302 29230 raft_consensus.cc:2272] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:40.110028 29230 ts_tablet_manager.cc:1916] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251210 21:06:40.114650 29230 ts_tablet_manager.cc:1929] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251210 21:06:40.114773 29230 log.cc:1199] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/bdafef28eb994416a18056a1954371bf
I20251210 21:06:40.115021 29230 ts_tablet_manager.cc:1950] T bdafef28eb994416a18056a1954371bf P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting consensus metadata
I20251210 21:06:40.116066 28659 catalog_manager.cc:4985] TS 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): tablet bdafef28eb994416a18056a1954371bf (table dugtrio [id=bbb03edea6be4fad9547146b036cb8f2]) successfully deleted
I20251210 21:06:40.301503 29231 ts_tablet_manager.cc:933] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Initiating tablet copy from peer 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:40.302059 29231 tablet_copy_client.cc:323] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:33303
I20251210 21:06:40.308177 28861 tablet_copy_service.cc:140] P 00cefb4b862b4514af558a61c220754f: Received BeginTabletCopySession request for tablet 15715890f89e4598805c5cc2ff489db3 from peer 6f9f3e8755674168af3060a746b58c24 ({username='slave'} at 127.0.0.1:41172)
I20251210 21:06:40.308384 28861 tablet_copy_service.cc:161] P 00cefb4b862b4514af558a61c220754f: Beginning new tablet copy session on tablet 15715890f89e4598805c5cc2ff489db3 from peer 6f9f3e8755674168af3060a746b58c24 at {username='slave'} at 127.0.0.1:41172: session id = 6f9f3e8755674168af3060a746b58c24-15715890f89e4598805c5cc2ff489db3
I20251210 21:06:40.309348 29234 ts_tablet_manager.cc:933] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Initiating tablet copy from peer 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:40.309933 29234 tablet_copy_client.cc:323] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:33303
I20251210 21:06:40.310596 28860 tablet_copy_service.cc:140] P 00cefb4b862b4514af558a61c220754f: Received BeginTabletCopySession request for tablet 18db8ba4df094f5f8e27ae62a446c78a from peer 6f9f3e8755674168af3060a746b58c24 ({username='slave'} at 127.0.0.1:41172)
I20251210 21:06:40.310719 28860 tablet_copy_service.cc:161] P 00cefb4b862b4514af558a61c220754f: Beginning new tablet copy session on tablet 18db8ba4df094f5f8e27ae62a446c78a from peer 6f9f3e8755674168af3060a746b58c24 at {username='slave'} at 127.0.0.1:41172: session id = 6f9f3e8755674168af3060a746b58c24-18db8ba4df094f5f8e27ae62a446c78a
I20251210 21:06:40.310902 28861 tablet_copy_source_session.cc:215] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:40.312140 29231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15715890f89e4598805c5cc2ff489db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:40.312954 28860 tablet_copy_source_session.cc:215] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:40.313937 29234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18db8ba4df094f5f8e27ae62a446c78a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:40.317574 29231 tablet_copy_client.cc:806] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:40.317809 29231 tablet_copy_client.cc:670] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:40.319417 29231 tablet_copy_client.cc:538] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:40.320042 29234 tablet_copy_client.cc:806] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:40.320240 29234 tablet_copy_client.cc:670] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:40.321844 29234 tablet_copy_client.cc:538] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:40.322793 29231 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Bootstrap starting.
I20251210 21:06:40.323513 29198 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:40.325055 29234 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Bootstrap starting.
I20251210 21:06:40.328989 29231 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:40.329357 29231 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Bootstrap complete.
I20251210 21:06:40.329578 29231 ts_tablet_manager.cc:1403] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.007s
I20251210 21:06:40.330199 29231 raft_consensus.cc:359] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.330374 29231 raft_consensus.cc:740] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f9f3e8755674168af3060a746b58c24, State: Initialized, Role: LEARNER
I20251210 21:06:40.330538 29231 consensus_queue.cc:260] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.330600 29234 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:40.330955 29234 tablet_bootstrap.cc:492] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Bootstrap complete.
I20251210 21:06:40.331211 29234 ts_tablet_manager.cc:1403] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.001s
I20251210 21:06:40.331241 29129 heartbeater.cc:499] Master 127.21.127.126:44701 was elected leader, sending a full tablet report...
I20251210 21:06:40.331277 29231 ts_tablet_manager.cc:1434] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251210 21:06:40.331938 28860 tablet_copy_service.cc:342] P 00cefb4b862b4514af558a61c220754f: Request end of tablet copy session 6f9f3e8755674168af3060a746b58c24-15715890f89e4598805c5cc2ff489db3 received from {username='slave'} at 127.0.0.1:41172
I20251210 21:06:40.332039 28860 tablet_copy_service.cc:434] P 00cefb4b862b4514af558a61c220754f: ending tablet copy session 6f9f3e8755674168af3060a746b58c24-15715890f89e4598805c5cc2ff489db3 on tablet 15715890f89e4598805c5cc2ff489db3 with peer 6f9f3e8755674168af3060a746b58c24
I20251210 21:06:40.332515 29234 raft_consensus.cc:359] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.332713 29234 raft_consensus.cc:740] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f9f3e8755674168af3060a746b58c24, State: Initialized, Role: LEARNER
I20251210 21:06:40.332897 29234 consensus_queue.cc:260] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.333453 29234 ts_tablet_manager.cc:1434] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251210 21:06:40.334026 28860 tablet_copy_service.cc:342] P 00cefb4b862b4514af558a61c220754f: Request end of tablet copy session 6f9f3e8755674168af3060a746b58c24-18db8ba4df094f5f8e27ae62a446c78a received from {username='slave'} at 127.0.0.1:41172
I20251210 21:06:40.334226 28860 tablet_copy_service.cc:434] P 00cefb4b862b4514af558a61c220754f: ending tablet copy session 6f9f3e8755674168af3060a746b58c24-18db8ba4df094f5f8e27ae62a446c78a on tablet 18db8ba4df094f5f8e27ae62a446c78a with peer 6f9f3e8755674168af3060a746b58c24
I20251210 21:06:40.344799 29234 ts_tablet_manager.cc:933] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Initiating tablet copy from peer ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:40.345459 29234 tablet_copy_client.cc:323] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:41227
I20251210 21:06:40.351457 28935 tablet_copy_service.cc:140] P ed0eda03e402411db17cfa89ad03980b: Received BeginTabletCopySession request for tablet 94d66b996d8742b192d3ab2b22decb5d from peer 6f9f3e8755674168af3060a746b58c24 ({username='slave'} at 127.0.0.1:39758)
I20251210 21:06:40.351639 28935 tablet_copy_service.cc:161] P ed0eda03e402411db17cfa89ad03980b: Beginning new tablet copy session on tablet 94d66b996d8742b192d3ab2b22decb5d from peer 6f9f3e8755674168af3060a746b58c24 at {username='slave'} at 127.0.0.1:39758: session id = 6f9f3e8755674168af3060a746b58c24-94d66b996d8742b192d3ab2b22decb5d
I20251210 21:06:40.352568 29198 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:40.354874 28935 tablet_copy_source_session.cc:215] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:40.356137 29234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94d66b996d8742b192d3ab2b22decb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:40.361335 29234 tablet_copy_client.cc:806] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:40.361531 29234 tablet_copy_client.cc:670] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:40.362962 29234 tablet_copy_client.cc:538] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:40.365449 29234 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Bootstrap starting.
I20251210 21:06:40.370153 29234 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:40.370440 29234 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Bootstrap complete.
I20251210 21:06:40.370635 29234 ts_tablet_manager.cc:1403] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20251210 21:06:40.371228 29234 raft_consensus.cc:359] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.371392 29234 raft_consensus.cc:740] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f9f3e8755674168af3060a746b58c24, State: Initialized, Role: LEARNER
I20251210 21:06:40.371575 29234 consensus_queue.cc:260] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.372040 29234 ts_tablet_manager.cc:1434] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251210 21:06:40.372807 28935 tablet_copy_service.cc:342] P ed0eda03e402411db17cfa89ad03980b: Request end of tablet copy session 6f9f3e8755674168af3060a746b58c24-94d66b996d8742b192d3ab2b22decb5d received from {username='slave'} at 127.0.0.1:39758
I20251210 21:06:40.372905 28935 tablet_copy_service.cc:434] P ed0eda03e402411db17cfa89ad03980b: ending tablet copy session 6f9f3e8755674168af3060a746b58c24-94d66b996d8742b192d3ab2b22decb5d on tablet 94d66b996d8742b192d3ab2b22decb5d with peer 6f9f3e8755674168af3060a746b58c24
I20251210 21:06:40.397300 29198 raft_consensus.cc:2955] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:40.443495 29234 ts_tablet_manager.cc:933] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Initiating tablet copy from peer ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:40.444010 29234 tablet_copy_client.cc:323] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:41227
I20251210 21:06:40.444612 28935 tablet_copy_service.cc:140] P ed0eda03e402411db17cfa89ad03980b: Received BeginTabletCopySession request for tablet 8dcdcf8eac50460b83c74ee9a90520e7 from peer 6f9f3e8755674168af3060a746b58c24 ({username='slave'} at 127.0.0.1:39758)
I20251210 21:06:40.444733 28935 tablet_copy_service.cc:161] P ed0eda03e402411db17cfa89ad03980b: Beginning new tablet copy session on tablet 8dcdcf8eac50460b83c74ee9a90520e7 from peer 6f9f3e8755674168af3060a746b58c24 at {username='slave'} at 127.0.0.1:39758: session id = 6f9f3e8755674168af3060a746b58c24-8dcdcf8eac50460b83c74ee9a90520e7
I20251210 21:06:40.446607 28935 tablet_copy_source_session.cc:215] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:40.447651 29234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dcdcf8eac50460b83c74ee9a90520e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:40.452322 29234 tablet_copy_client.cc:806] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:40.452538 29234 tablet_copy_client.cc:670] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:40.453428 29198 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:40.453887 29234 tablet_copy_client.cc:538] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:40.464986 29234 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Bootstrap starting.
I20251210 21:06:40.470605 29234 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:40.470947 29234 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Bootstrap complete.
I20251210 21:06:40.471171 29234 ts_tablet_manager.cc:1403] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251210 21:06:40.471819 29234 raft_consensus.cc:359] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.471971 29234 raft_consensus.cc:740] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f9f3e8755674168af3060a746b58c24, State: Initialized, Role: LEARNER
I20251210 21:06:40.472146 29234 consensus_queue.cc:260] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:40.472586 29234 ts_tablet_manager.cc:1434] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251210 21:06:40.473536 28935 tablet_copy_service.cc:342] P ed0eda03e402411db17cfa89ad03980b: Request end of tablet copy session 6f9f3e8755674168af3060a746b58c24-8dcdcf8eac50460b83c74ee9a90520e7 received from {username='slave'} at 127.0.0.1:39758
I20251210 21:06:40.473649 28935 tablet_copy_service.cc:434] P ed0eda03e402411db17cfa89ad03980b: ending tablet copy session 6f9f3e8755674168af3060a746b58c24-8dcdcf8eac50460b83c74ee9a90520e7 on tablet 8dcdcf8eac50460b83c74ee9a90520e7 with peer 6f9f3e8755674168af3060a746b58c24
W20251210 21:06:40.479944 28722 auto_rebalancer.cc:254] failed to send replica move request: Invalid argument: Server with UUID 6f9f3e8755674168af3060a746b58c24 is already a member of the config. RaftConfig: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
W20251210 21:06:40.481576 28722 auto_rebalancer.cc:663] Could not move replica: Illegal state: 18db8ba4df094f5f8e27ae62a446c78a: source replica ed0eda03e402411db17cfa89ad03980b does not have REPLACE attribute set
W20251210 21:06:40.481673 28722 auto_rebalancer.cc:264] scheduled replica move failed to complete: Illegal state: 18db8ba4df094f5f8e27ae62a446c78a: source replica ed0eda03e402411db17cfa89ad03980b does not have REPLACE attribute set
W20251210 21:06:40.483038 28722 auto_rebalancer.cc:663] Could not move replica: Illegal state: 8dcdcf8eac50460b83c74ee9a90520e7: source replica 0e0cab8636c34a3eabffe2495f82e1ae does not have REPLACE attribute set
W20251210 21:06:40.483105 28722 auto_rebalancer.cc:264] scheduled replica move failed to complete: Illegal state: 8dcdcf8eac50460b83c74ee9a90520e7: source replica 0e0cab8636c34a3eabffe2495f82e1ae does not have REPLACE attribute set
W20251210 21:06:40.484270 28722 auto_rebalancer.cc:663] Could not move replica: Illegal state: 94d66b996d8742b192d3ab2b22decb5d: source replica 00cefb4b862b4514af558a61c220754f does not have REPLACE attribute set
W20251210 21:06:40.484336 28722 auto_rebalancer.cc:264] scheduled replica move failed to complete: Illegal state: 94d66b996d8742b192d3ab2b22decb5d: source replica 00cefb4b862b4514af558a61c220754f does not have REPLACE attribute set
I20251210 21:06:40.742192 28924 consensus_queue.cc:237] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:40.744004 28851 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:40.744024 29198 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:40.744563 29246 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:40.744750 29250 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:40.746865 29022 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.747530 28851 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.747658 29198 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.750378 28659 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 94d66b996d8742b192d3ab2b22decb5d with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:40.752437 28674 catalog_manager.cc:5654] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f reported cstate change: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.766860 29094 tablet_service.cc:1552] Processing DeleteTablet for tablet 94d66b996d8742b192d3ab2b22decb5d with delete_type TABLET_DATA_TOMBSTONED (TS 6f9f3e8755674168af3060a746b58c24 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:39454
I20251210 21:06:40.767606 29256 tablet_replica.cc:333] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: stopping tablet replica
I20251210 21:06:40.767784 29256 raft_consensus.cc:2243] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus shutting down.
I20251210 21:06:40.767936 29256 raft_consensus.cc:2272] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus is shut down!
I20251210 21:06:40.768615 29256 ts_tablet_manager.cc:1916] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:06:40.773543 29256 ts_tablet_manager.cc:1929] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20251210 21:06:40.773662 29256 log.cc:1199] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root/wals/94d66b996d8742b192d3ab2b22decb5d
I20251210 21:06:40.774435 28660 catalog_manager.cc:4985] TS 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): tablet 94d66b996d8742b192d3ab2b22decb5d (table test-workload [id=e2e260b5487e4123bab2b5c51b620238]) successfully deleted
I20251210 21:06:40.804342 29104 raft_consensus.cc:1217] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251210 21:06:40.812428 28851 consensus_queue.cc:237] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:40.814249 29198 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:40.814249 28924 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:40.814776 29257 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:40.814982 29258 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:40.817365 29044 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.818028 29198 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.818040 28924 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.821842 28661 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 15715890f89e4598805c5cc2ff489db3 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:40.822655 28674 catalog_manager.cc:5654] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f reported cstate change: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } }
I20251210 21:06:40.826354 29094 tablet_service.cc:1552] Processing DeleteTablet for tablet 15715890f89e4598805c5cc2ff489db3 with delete_type TABLET_DATA_TOMBSTONED (TS 6f9f3e8755674168af3060a746b58c24 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:39454
I20251210 21:06:40.826862 29256 tablet_replica.cc:333] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: stopping tablet replica
I20251210 21:06:40.827035 29256 raft_consensus.cc:2243] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus shutting down.
I20251210 21:06:40.827209 29256 raft_consensus.cc:2272] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus is shut down!
I20251210 21:06:40.827821 29256 ts_tablet_manager.cc:1916] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:06:40.832468 29256 ts_tablet_manager.cc:1929] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20251210 21:06:40.832607 29256 log.cc:1199] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root/wals/15715890f89e4598805c5cc2ff489db3
I20251210 21:06:40.833336 28660 catalog_manager.cc:4985] TS 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): tablet 15715890f89e4598805c5cc2ff489db3 (table test-workload [id=e2e260b5487e4123bab2b5c51b620238]) successfully deleted
I20251210 21:06:40.957930 28924 consensus_queue.cc:237] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:40.959803 29198 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:40.959823 28851 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:40.960282 29250 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:40.960467 29246 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:40.962476 29250 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.963094 29198 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.963728 28851 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.966358 28659 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 8dcdcf8eac50460b83c74ee9a90520e7 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:40.967928 28672 catalog_manager.cc:5654] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f reported cstate change: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:40.971361 29094 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dcdcf8eac50460b83c74ee9a90520e7 with delete_type TABLET_DATA_TOMBSTONED (TS 6f9f3e8755674168af3060a746b58c24 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:39454
I20251210 21:06:40.971709 29256 tablet_replica.cc:333] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: stopping tablet replica
I20251210 21:06:40.971917 29256 raft_consensus.cc:2243] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus shutting down.
I20251210 21:06:40.972148 29256 raft_consensus.cc:2272] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus is shut down!
I20251210 21:06:40.972795 29256 ts_tablet_manager.cc:1916] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:06:40.977689 29256 ts_tablet_manager.cc:1929] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20251210 21:06:40.977797 29256 log.cc:1199] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root/wals/8dcdcf8eac50460b83c74ee9a90520e7
I20251210 21:06:40.978358 28660 catalog_manager.cc:4985] TS 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): tablet 8dcdcf8eac50460b83c74ee9a90520e7 (table test-workload [id=e2e260b5487e4123bab2b5c51b620238]) successfully deleted
I20251210 21:06:40.998037 28851 consensus_queue.cc:237] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } }
I20251210 21:06:40.999840 28924 raft_consensus.cc:1275] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:40.999891 29198 raft_consensus.cc:1275] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251210 21:06:41.000334 29258 consensus_queue.cc:1048] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:41.000526 29257 consensus_queue.cc:1048] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251210 21:06:41.002542 29258 raft_consensus.cc:2955] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:41.003208 29198 raft_consensus.cc:2955] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:41.003571 28924 raft_consensus.cc:2955] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:41.006454 28661 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 18db8ba4df094f5f8e27ae62a446c78a with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:41.007818 28674 catalog_manager.cc:5654] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae reported cstate change: config changed from index 2 to 3, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) evicted. New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } }
I20251210 21:06:41.011391 29094 tablet_service.cc:1552] Processing DeleteTablet for tablet 18db8ba4df094f5f8e27ae62a446c78a with delete_type TABLET_DATA_TOMBSTONED (TS 6f9f3e8755674168af3060a746b58c24 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:39454
I20251210 21:06:41.011745 29256 tablet_replica.cc:333] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: stopping tablet replica
I20251210 21:06:41.011941 29256 raft_consensus.cc:2243] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus shutting down.
I20251210 21:06:41.012115 29256 raft_consensus.cc:2272] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Raft consensus is shut down!
I20251210 21:06:41.012738 29256 ts_tablet_manager.cc:1916] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:06:41.017052 29256 ts_tablet_manager.cc:1929] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20251210 21:06:41.017148 29256 log.cc:1199] T 18db8ba4df094f5f8e27ae62a446c78a P 6f9f3e8755674168af3060a746b58c24: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-3-root/wals/18db8ba4df094f5f8e27ae62a446c78a
I20251210 21:06:41.017741 28660 catalog_manager.cc:4985] TS 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): tablet 18db8ba4df094f5f8e27ae62a446c78a (table test-workload [id=e2e260b5487e4123bab2b5c51b620238]) successfully deleted
I20251210 21:06:41.493784 28924 consensus_queue.cc:237] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:41.495922 28851 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251210 21:06:41.495965 28890 consensus_peers.cc:597] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b -> Peer 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): Couldn't send request to peer 6f9f3e8755674168af3060a746b58c24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:41.496177 29198 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:06:41.496549 29022 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:41.496815 29265 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:41.499156 29022 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.499617 28851 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.503098 29198 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.504873 28672 catalog_manager.cc:5654] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:41.507817 28851 consensus_queue.cc:237] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
W20251210 21:06:41.510965 28816 consensus_peers.cc:597] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f -> Peer 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): Couldn't send request to peer 6f9f3e8755674168af3060a746b58c24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:41.511307 29198 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:06:41.511359 28924 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:06:41.511785 29268 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:41.512148 29257 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:41.514245 29268 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.514616 28924 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.514811 29198 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.519163 28672 catalog_manager.cc:5654] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.519945 28924 consensus_queue.cc:237] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:41.521926 28851 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:06:41.522046 29198 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251210 21:06:41.522378 29022 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251210 21:06:41.522611 29265 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20251210 21:06:41.523097 28890 consensus_peers.cc:597] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b -> Peer 6f9f3e8755674168af3060a746b58c24 (127.21.127.68:39083): Couldn't send request to peer 6f9f3e8755674168af3060a746b58c24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:41.524708 29022 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.525255 28851 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.526646 29198 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } } }
I20251210 21:06:41.529711 28674 catalog_manager.cc:5654] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index 3 to 4, NON_VOTER 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) added. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251210 21:06:41.913391 29270 ts_tablet_manager.cc:933] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Initiating tablet copy from peer ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:41.914037 29270 tablet_copy_client.cc:323] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:41227
I20251210 21:06:41.914637 28935 tablet_copy_service.cc:140] P ed0eda03e402411db17cfa89ad03980b: Received BeginTabletCopySession request for tablet 94d66b996d8742b192d3ab2b22decb5d from peer 6f9f3e8755674168af3060a746b58c24 ({username='slave'} at 127.0.0.1:39758)
I20251210 21:06:41.914774 28935 tablet_copy_service.cc:161] P ed0eda03e402411db17cfa89ad03980b: Beginning new tablet copy session on tablet 94d66b996d8742b192d3ab2b22decb5d from peer 6f9f3e8755674168af3060a746b58c24 at {username='slave'} at 127.0.0.1:39758: session id = 6f9f3e8755674168af3060a746b58c24-94d66b996d8742b192d3ab2b22decb5d
I20251210 21:06:41.917225 28935 tablet_copy_source_session.cc:215] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:41.918120 29270 ts_tablet_manager.cc:1916] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Deleting tablet data with delete state TABLET_DATA_COPYING
I20251210 21:06:41.924384 29270 ts_tablet_manager.cc:1929] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20251210 21:06:41.924692 29270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94d66b996d8742b192d3ab2b22decb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:41.927659 29270 tablet_copy_client.cc:806] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:41.927917 29270 tablet_copy_client.cc:670] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:41.929574 29270 tablet_copy_client.cc:538] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:41.932438 29270 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Bootstrap starting.
I20251210 21:06:41.938508 29270 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:41.938845 29270 tablet_bootstrap.cc:492] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Bootstrap complete.
I20251210 21:06:41.939200 29270 ts_tablet_manager.cc:1403] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20251210 21:06:41.939910 29270 raft_consensus.cc:359] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:41.940147 29270 raft_consensus.cc:740] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f9f3e8755674168af3060a746b58c24, State: Initialized, Role: LEARNER
I20251210 21:06:41.940331 29270 consensus_queue.cc:260] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:41.941126 29270 ts_tablet_manager.cc:1434] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20251210 21:06:41.941872 28935 tablet_copy_service.cc:342] P ed0eda03e402411db17cfa89ad03980b: Request end of tablet copy session 6f9f3e8755674168af3060a746b58c24-94d66b996d8742b192d3ab2b22decb5d received from {username='slave'} at 127.0.0.1:39758
I20251210 21:06:41.941995 28935 tablet_copy_service.cc:434] P ed0eda03e402411db17cfa89ad03980b: ending tablet copy session 6f9f3e8755674168af3060a746b58c24-94d66b996d8742b192d3ab2b22decb5d on tablet 94d66b996d8742b192d3ab2b22decb5d with peer 6f9f3e8755674168af3060a746b58c24
I20251210 21:06:41.958120 29270 ts_tablet_manager.cc:933] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Initiating tablet copy from peer 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303)
I20251210 21:06:41.958773 29270 tablet_copy_client.cc:323] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.66:33303
I20251210 21:06:41.959319 28860 tablet_copy_service.cc:140] P 00cefb4b862b4514af558a61c220754f: Received BeginTabletCopySession request for tablet 15715890f89e4598805c5cc2ff489db3 from peer 6f9f3e8755674168af3060a746b58c24 ({username='slave'} at 127.0.0.1:41172)
I20251210 21:06:41.959489 28860 tablet_copy_service.cc:161] P 00cefb4b862b4514af558a61c220754f: Beginning new tablet copy session on tablet 15715890f89e4598805c5cc2ff489db3 from peer 6f9f3e8755674168af3060a746b58c24 at {username='slave'} at 127.0.0.1:41172: session id = 6f9f3e8755674168af3060a746b58c24-15715890f89e4598805c5cc2ff489db3
I20251210 21:06:41.961700 28860 tablet_copy_source_session.cc:215] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:41.962615 29270 ts_tablet_manager.cc:1916] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Deleting tablet data with delete state TABLET_DATA_COPYING
I20251210 21:06:41.968420 29270 ts_tablet_manager.cc:1929] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20251210 21:06:41.968685 29270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15715890f89e4598805c5cc2ff489db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:41.971614 29270 tablet_copy_client.cc:806] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:41.971889 29270 tablet_copy_client.cc:670] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:41.973502 29270 tablet_copy_client.cc:538] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:41.976655 29270 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Bootstrap starting.
I20251210 21:06:41.983528 29270 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:41.983909 29270 tablet_bootstrap.cc:492] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Bootstrap complete.
I20251210 21:06:41.984145 29270 ts_tablet_manager.cc:1403] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20251210 21:06:41.984848 29270 raft_consensus.cc:359] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:41.985057 29270 raft_consensus.cc:740] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f9f3e8755674168af3060a746b58c24, State: Initialized, Role: LEARNER
I20251210 21:06:41.985235 29270 consensus_queue.cc:260] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:41.985728 29270 ts_tablet_manager.cc:1434] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251210 21:06:41.986320 28860 tablet_copy_service.cc:342] P 00cefb4b862b4514af558a61c220754f: Request end of tablet copy session 6f9f3e8755674168af3060a746b58c24-15715890f89e4598805c5cc2ff489db3 received from {username='slave'} at 127.0.0.1:41172
I20251210 21:06:41.986423 28860 tablet_copy_service.cc:434] P 00cefb4b862b4514af558a61c220754f: ending tablet copy session 6f9f3e8755674168af3060a746b58c24-15715890f89e4598805c5cc2ff489db3 on tablet 15715890f89e4598805c5cc2ff489db3 with peer 6f9f3e8755674168af3060a746b58c24
I20251210 21:06:42.006656 29270 ts_tablet_manager.cc:933] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Initiating tablet copy from peer ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227)
I20251210 21:06:42.007745 29270 tablet_copy_client.cc:323] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Beginning tablet copy session from remote peer at address 127.21.127.67:41227
I20251210 21:06:42.008327 28935 tablet_copy_service.cc:140] P ed0eda03e402411db17cfa89ad03980b: Received BeginTabletCopySession request for tablet 8dcdcf8eac50460b83c74ee9a90520e7 from peer 6f9f3e8755674168af3060a746b58c24 ({username='slave'} at 127.0.0.1:39758)
I20251210 21:06:42.008457 28935 tablet_copy_service.cc:161] P ed0eda03e402411db17cfa89ad03980b: Beginning new tablet copy session on tablet 8dcdcf8eac50460b83c74ee9a90520e7 from peer 6f9f3e8755674168af3060a746b58c24 at {username='slave'} at 127.0.0.1:39758: session id = 6f9f3e8755674168af3060a746b58c24-8dcdcf8eac50460b83c74ee9a90520e7
I20251210 21:06:42.010882 28935 tablet_copy_source_session.cc:215] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: Tablet Copy: opened 0 blocks and 1 log segments
I20251210 21:06:42.011677 29270 ts_tablet_manager.cc:1916] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Deleting tablet data with delete state TABLET_DATA_COPYING
I20251210 21:06:42.015944 29270 ts_tablet_manager.cc:1929] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20251210 21:06:42.016177 29270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dcdcf8eac50460b83c74ee9a90520e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251210 21:06:42.018435 29270 tablet_copy_client.cc:806] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 0 data blocks...
I20251210 21:06:42.018672 29270 tablet_copy_client.cc:670] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Starting download of 1 WAL segments...
I20251210 21:06:42.020815 29270 tablet_copy_client.cc:538] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251210 21:06:42.023649 29270 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Bootstrap starting.
I20251210 21:06:42.029850 29270 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251210 21:06:42.030174 29270 tablet_bootstrap.cc:492] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Bootstrap complete.
I20251210 21:06:42.030407 29270 ts_tablet_manager.cc:1403] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251210 21:06:42.031050 29270 raft_consensus.cc:359] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:42.031235 29270 raft_consensus.cc:740] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f9f3e8755674168af3060a746b58c24, State: Initialized, Role: LEARNER
I20251210 21:06:42.031463 29270 consensus_queue.cc:260] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: NON_VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: true } }
I20251210 21:06:42.032049 29270 ts_tablet_manager.cc:1434] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251210 21:06:42.032733 28935 tablet_copy_service.cc:342] P ed0eda03e402411db17cfa89ad03980b: Request end of tablet copy session 6f9f3e8755674168af3060a746b58c24-8dcdcf8eac50460b83c74ee9a90520e7 received from {username='slave'} at 127.0.0.1:39758
I20251210 21:06:42.032835 28935 tablet_copy_service.cc:434] P ed0eda03e402411db17cfa89ad03980b: ending tablet copy session 6f9f3e8755674168af3060a746b58c24-8dcdcf8eac50460b83c74ee9a90520e7 on tablet 8dcdcf8eac50460b83c74ee9a90520e7 with peer 6f9f3e8755674168af3060a746b58c24
I20251210 21:06:42.388592 29104 raft_consensus.cc:1217] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251210 21:06:42.446712 29104 raft_consensus.cc:1217] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251210 21:06:42.534855 29104 raft_consensus.cc:1217] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251210 21:06:42.873028 29279 raft_consensus.cc:1064] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: attempting to promote NON_VOTER 6f9f3e8755674168af3060a746b58c24 to VOTER
I20251210 21:06:42.873797 29279 consensus_queue.cc:237] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } }
I20251210 21:06:42.876084 29104 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:42.876175 28851 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:42.876175 29198 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:42.876652 29279 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:42.877008 29278 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:42.877494 29022 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:42.880597 29022 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.881132 28851 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.881804 29198 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.884394 29104 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.885121 28673 catalog_manager.cc:5654] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:42.889539 29283 raft_consensus.cc:1064] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: attempting to promote NON_VOTER 6f9f3e8755674168af3060a746b58c24 to VOTER
I20251210 21:06:42.890244 29283 consensus_queue.cc:237] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } }
I20251210 21:06:42.892946 28851 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:42.893213 29103 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:42.893515 29279 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:42.893754 29278 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:42.894101 29198 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:42.894670 29279 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:42.909159 29278 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.912498 29198 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.913364 29104 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.916769 28851 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.919011 28671 catalog_manager.cc:5654] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:42.920809 28925 consensus_queue.cc:237] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } }
I20251210 21:06:42.922730 28851 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251210 21:06:42.922752 29104 raft_consensus.cc:1275] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251210 21:06:42.924125 29022 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251210 21:06:42.924818 28924 consensus_queue.cc:237] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } }
I20251210 21:06:42.925324 29022 consensus_queue.cc:1048] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251210 21:06:42.926743 29103 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251210 21:06:42.927416 29198 raft_consensus.cc:1275] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer ed0eda03e402411db17cfa89ad03980b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251210 21:06:42.927822 29022 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251210 21:06:42.927845 29278 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.928560 29022 consensus_queue.cc:1048] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251210 21:06:42.928584 28851 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.929575 29103 raft_consensus.cc:2955] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.930078 29283 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 00cefb4b862b4514af558a61c220754f (127.21.127.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.931959 28659 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 8dcdcf8eac50460b83c74ee9a90520e7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:42.932461 29104 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 00cefb4b862b4514af558a61c220754f (127.21.127.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.933116 28672 catalog_manager.cc:5654] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b reported cstate change: config changed from index 5 to 6, VOTER 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65) evicted. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:42.936405 28659 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 94d66b996d8742b192d3ab2b22decb5d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:42.937700 29188 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dcdcf8eac50460b83c74ee9a90520e7 with delete_type TABLET_DATA_TOMBSTONED (TS 0e0cab8636c34a3eabffe2495f82e1ae not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:40878
I20251210 21:06:42.938335 29296 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:42.938555 29296 raft_consensus.cc:2243] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:42.938897 29296 raft_consensus.cc:2272] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:42.939972 29296 ts_tablet_manager.cc:1916] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:06:42.940896 28674 catalog_manager.cc:5654] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 reported cstate change: config changed from index 5 to 6, VOTER 00cefb4b862b4514af558a61c220754f (127.21.127.66) evicted. New cstate: current_term: 1 leader_uuid: "ed0eda03e402411db17cfa89ad03980b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.941921 29198 raft_consensus.cc:2955] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 00cefb4b862b4514af558a61c220754f (127.21.127.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:42.946480 29296 ts_tablet_manager.cc:1929] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251210 21:06:42.946599 29296 log.cc:1199] T 8dcdcf8eac50460b83c74ee9a90520e7 P 0e0cab8636c34a3eabffe2495f82e1ae: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-0-root/wals/8dcdcf8eac50460b83c74ee9a90520e7
I20251210 21:06:42.947408 28659 catalog_manager.cc:4985] TS 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): tablet 8dcdcf8eac50460b83c74ee9a90520e7 (table test-workload [id=e2e260b5487e4123bab2b5c51b620238]) successfully deleted
I20251210 21:06:42.949199 28840 tablet_service.cc:1552] Processing DeleteTablet for tablet 94d66b996d8742b192d3ab2b22decb5d with delete_type TABLET_DATA_TOMBSTONED (TS 00cefb4b862b4514af558a61c220754f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41142
I20251210 21:06:42.949792 29297 tablet_replica.cc:333] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:42.949965 29297 raft_consensus.cc:2243] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:42.950294 29297 raft_consensus.cc:2272] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:42.952004 29297 ts_tablet_manager.cc:1916] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:06:42.957530 29297 ts_tablet_manager.cc:1929] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251210 21:06:42.957670 29297 log.cc:1199] T 94d66b996d8742b192d3ab2b22decb5d P 00cefb4b862b4514af558a61c220754f: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-1-root/wals/94d66b996d8742b192d3ab2b22decb5d
I20251210 21:06:42.958299 28661 catalog_manager.cc:4985] TS 00cefb4b862b4514af558a61c220754f (127.21.127.66:33303): tablet 94d66b996d8742b192d3ab2b22decb5d (table test-workload [id=e2e260b5487e4123bab2b5c51b620238]) successfully deleted
I20251210 21:06:43.097190 29298 raft_consensus.cc:1064] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: attempting to promote NON_VOTER 6f9f3e8755674168af3060a746b58c24 to VOTER
I20251210 21:06:43.098253 29298 consensus_queue.cc:237] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } }
I20251210 21:06:43.100410 29198 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:43.100445 28924 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:43.100931 29298 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:43.101246 29104 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 LEARNER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251210 21:06:43.101302 29299 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:43.102026 29299 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251210 21:06:43.114145 29301 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:43.114995 28924 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:43.117709 29198 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:43.117703 29104 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:43.118502 28672 catalog_manager.cc:5654] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f reported cstate change: config changed from index 4 to 5, 6f9f3e8755674168af3060a746b58c24 (127.21.127.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed0eda03e402411db17cfa89ad03980b" member_type: VOTER last_known_addr { host: "127.21.127.67" port: 41227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:43.124269 28851 consensus_queue.cc:237] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } }
I20251210 21:06:43.126624 29198 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251210 21:06:43.127292 29104 raft_consensus.cc:1275] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Refusing update from remote peer 00cefb4b862b4514af558a61c220754f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251210 21:06:43.127501 29298 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251210 21:06:43.129253 29298 consensus_queue.cc:1048] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251210 21:06:43.130241 29299 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ed0eda03e402411db17cfa89ad03980b (127.21.127.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:43.133039 29104 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ed0eda03e402411db17cfa89ad03980b (127.21.127.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:43.134779 28661 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 15715890f89e4598805c5cc2ff489db3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251210 21:06:43.135210 29198 raft_consensus.cc:2955] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ed0eda03e402411db17cfa89ad03980b (127.21.127.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } } }
I20251210 21:06:43.135756 28671 catalog_manager.cc:5654] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f reported cstate change: config changed from index 5 to 6, VOTER ed0eda03e402411db17cfa89ad03980b (127.21.127.67) evicted. New cstate: current_term: 1 leader_uuid: "00cefb4b862b4514af558a61c220754f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0e0cab8636c34a3eabffe2495f82e1ae" member_type: VOTER last_known_addr { host: "127.21.127.65" port: 43855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00cefb4b862b4514af558a61c220754f" member_type: VOTER last_known_addr { host: "127.21.127.66" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9f3e8755674168af3060a746b58c24" member_type: VOTER last_known_addr { host: "127.21.127.68" port: 39083 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251210 21:06:43.139276 28914 tablet_service.cc:1552] Processing DeleteTablet for tablet 15715890f89e4598805c5cc2ff489db3 with delete_type TABLET_DATA_TOMBSTONED (TS ed0eda03e402411db17cfa89ad03980b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39718
I20251210 21:06:43.139994 29305 tablet_replica.cc:333] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:43.140329 29305 raft_consensus.cc:2243] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.140588 29305 raft_consensus.cc:2272] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.141294 29305 ts_tablet_manager.cc:1916] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251210 21:06:43.145576 29305 ts_tablet_manager.cc:1929] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251210 21:06:43.145670 29305 log.cc:1199] T 15715890f89e4598805c5cc2ff489db3 P ed0eda03e402411db17cfa89ad03980b: Deleting WAL directory at /tmp/dist-test-taskJOcOIB/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765400617643276-22013-0/minicluster-data/ts-2-root/wals/15715890f89e4598805c5cc2ff489db3
I20251210 21:06:43.146266 28659 catalog_manager.cc:4985] TS ed0eda03e402411db17cfa89ad03980b (127.21.127.67:41227): tablet 15715890f89e4598805c5cc2ff489db3 (table test-workload [id=e2e260b5487e4123bab2b5c51b620238]) successfully deleted
I20251210 21:06:43.489689 22013 tablet_server.cc:178] TabletServer@127.21.127.65:43855 shutting down...
I20251210 21:06:43.497750 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:43.498076 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:43.498270 22013 raft_consensus.cc:2243] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.498525 22013 raft_consensus.cc:2272] T 18db8ba4df094f5f8e27ae62a446c78a P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.499151 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:43.499310 22013 raft_consensus.cc:2243] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.499518 22013 raft_consensus.cc:2272] T 94d66b996d8742b192d3ab2b22decb5d P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.500219 22013 tablet_replica.cc:333] stopping tablet replica
I20251210 21:06:43.500340 22013 raft_consensus.cc:2243] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.500603 22013 raft_consensus.cc:2272] T 15715890f89e4598805c5cc2ff489db3 P 0e0cab8636c34a3eabffe2495f82e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.515575 22013 tablet_server.cc:195] TabletServer@127.21.127.65:43855 shutdown complete.
I20251210 21:06:43.520226 22013 tablet_server.cc:178] TabletServer@127.21.127.66:0 shutting down...
W20251210 21:06:43.528215 28889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111) [suppressed 38 similar messages]
I20251210 21:06:43.528980 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:43.529228 22013 tablet_replica.cc:333] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:43.529474 22013 raft_consensus.cc:2243] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus shutting down.
W20251210 21:06:43.529718 28889 consensus_peers.cc:597] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b -> Peer 0e0cab8636c34a3eabffe2495f82e1ae (127.21.127.65:43855): Couldn't send request to peer 0e0cab8636c34a3eabffe2495f82e1ae. Status: Network error: Client connection negotiation failed: client connection to 127.21.127.65:43855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251210 21:06:43.529894 22013 raft_consensus.cc:2272] T 8dcdcf8eac50460b83c74ee9a90520e7 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.530696 22013 tablet_replica.cc:333] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:43.530815 22013 raft_consensus.cc:2243] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:43.531071 22013 raft_consensus.cc:2272] T 18db8ba4df094f5f8e27ae62a446c78a P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.531579 22013 tablet_replica.cc:333] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f: stopping tablet replica
I20251210 21:06:43.531684 22013 raft_consensus.cc:2243] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:43.531905 22013 raft_consensus.cc:2272] T 15715890f89e4598805c5cc2ff489db3 P 00cefb4b862b4514af558a61c220754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.559860 22013 tablet_server.cc:195] TabletServer@127.21.127.66:0 shutdown complete.
I20251210 21:06:43.564882 22013 tablet_server.cc:178] TabletServer@127.21.127.67:0 shutting down...
I20251210 21:06:43.573474 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:43.573882 22013 tablet_replica.cc:333] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:43.574051 22013 raft_consensus.cc:2243] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:43.574390 22013 raft_consensus.cc:2272] T 94d66b996d8742b192d3ab2b22decb5d P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.575191 22013 tablet_replica.cc:333] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:43.575331 22013 raft_consensus.cc:2243] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.575567 22013 raft_consensus.cc:2272] T 18db8ba4df094f5f8e27ae62a446c78a P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.576061 22013 tablet_replica.cc:333] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b: stopping tablet replica
I20251210 21:06:43.576179 22013 raft_consensus.cc:2243] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:43.576440 22013 raft_consensus.cc:2272] T 8dcdcf8eac50460b83c74ee9a90520e7 P ed0eda03e402411db17cfa89ad03980b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.591226 22013 tablet_server.cc:195] TabletServer@127.21.127.67:0 shutdown complete.
I20251210 21:06:43.596017 22013 tablet_server.cc:178] TabletServer@127.21.127.68:0 shutting down...
I20251210 21:06:43.607590 22013 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251210 21:06:43.607899 22013 tablet_replica.cc:333] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24: stopping tablet replica
I20251210 21:06:43.608103 22013 raft_consensus.cc:2243] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.608352 22013 raft_consensus.cc:2272] T 8dcdcf8eac50460b83c74ee9a90520e7 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.609194 22013 tablet_replica.cc:333] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24: stopping tablet replica
I20251210 21:06:43.609337 22013 raft_consensus.cc:2243] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.609535 22013 raft_consensus.cc:2272] T 94d66b996d8742b192d3ab2b22decb5d P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.610210 22013 tablet_replica.cc:333] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24: stopping tablet replica
I20251210 21:06:43.610339 22013 raft_consensus.cc:2243] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251210 21:06:43.610517 22013 raft_consensus.cc:2272] T 15715890f89e4598805c5cc2ff489db3 P 6f9f3e8755674168af3060a746b58c24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.625916 22013 tablet_server.cc:195] TabletServer@127.21.127.68:0 shutdown complete.
I20251210 21:06:43.640872 22013 master.cc:561] Master@127.21.127.126:44701 shutting down...
I20251210 21:06:43.648240 22013 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 1 LEADER]: Raft consensus shutting down.
I20251210 21:06:43.648485 22013 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251210 21:06:43.648591 22013 tablet_replica.cc:333] T 00000000000000000000000000000000 P 96baabc14c4f4e92b2892043f191b224: stopping tablet replica
I20251210 21:06:43.662842 22013 master.cc:583] Master@127.21.127.126:44701 shutdown complete.
[ OK ] AutoRebalancerTest.TestDeletedTables (7286 ms)
[----------] 14 tests from AutoRebalancerTest (185968 ms total)
[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (185969 ms total)
[ PASSED ] 13 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer
1 FAILED TEST
I20251210 21:06:43.675882 22013 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 7 messages since previous log ~3 seconds ago
I20251210 21:06:43.675973 22013 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 7 messages since previous log ~3 seconds ago
I20251210 21:06:43.676000 22013 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:4712: suppressed but not reported on 14 messages since previous log ~3 seconds ago
I20251210 21:06:43.676029 22013 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:5332: suppressed but not reported on 11 messages since previous log ~29 seconds ago
I20251210 21:06:43.676061 22013 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 8 messages since previous log ~172 seconds ago
I20251210 21:06:43.676147 22013 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 50 messages since previous log ~17 seconds ago